Build a valid robots.txt in seconds — allow or block crawlers, add disallow rules, a crawl-delay and your sitemap — then copy it straight to your site root.
The Robots.txt Generator writes a correctly-formatted robots.txt for you from a simple form. Choose to allow everything, block everything, or set custom allow/disallow rules per user-agent, add an optional crawl-delay, and point crawlers to your sitemap — the tool keeps the syntax valid so a stray character never blocks your whole site.
robots.txt tells search engine crawlers which parts of your site they may request. A single wrong line can deindex pages or expose paths you meant to hide, so getting the syntax right matters. This generator runs in your browser, produces copy-ready output, and you simply paste it into a file named robots.txt at your domain root.
One click for "allow all" or "block all", or set your own rules.
Add disallow/allow paths and a crawl-delay per user-agent.
Drop in your sitemap URL so crawlers find every page.
Pick a preset or add custom allow/disallow paths and your sitemap URL.
Grab the generated, validated robots.txt text.
Save it as robots.txt at https://yourdomain.com/robots.txt.