Robots.txt Generator
Build a robots.txt and see what each rule actually blocks before you publish it.
About this tool
List the paths you want to keep crawlers out of, one per line, and this builds the file — then tests the result against the same parser used to check live sites and shows what each rule really blocks.
That second step is the point. "Disallow: /" is one character away from "Disallow: /admin" and removes an entire site from crawling. Nothing about the finished file looks any different, which is why this shows the consequence rather than only the syntax.
Common questions
Does robots.txt stop a page appearing in search?
No. It stops a page being crawled. A blocked URL can still be listed — usually with no description — if other pages link to it. To keep a page out of results, allow crawling and serve a noindex directive instead.
Should I block my admin area?
It does nothing for security: robots.txt is public, so it advertises exactly the paths you would rather nobody looked at. Protect them with authentication. Blocking is worth it only to save crawl budget on large, pointless sections.
Why did my Disallow rule not work?
Usually a longer Allow rule also matched. Under RFC 9309 the longest matching pattern wins, so Allow: /blog/public/ beats Disallow: /blog/. The test below shows which rule actually decides each path.
Where does the file go?
The root of the domain, and nowhere else. https://example.com/robots.txt is read; https://example.com/blog/robots.txt is ignored entirely. Each subdomain needs its own.