Robots.txt Generator

Generate a robots.txt file to control which pages search engine crawlers can access.

Sitemap URL (optional)

Rule

User-agent

Allow paths (one per line)

Disallow paths (one per line)

Crawl Delay (seconds)

Generated robots.txt