seo
Processed locally in your browser
Robots.txt Generator
Generate standard robots.txt files for search engine crawlers with allow/disallow rules.
/api//admin/
How to Use Robots.txt Generator
1
Configure Directives
Select default permissions and add paths to block (e.g. /admin/, /private/).
2
Add Sitemap
Enter your full sitemap URL (e.g. https://example.com/sitemap.xml).
3
Download
Download the file and place it in your site's root directory.
Features & Capabilities
Pre-configured profiles for Googlebot, Bingbot, and universal (*)
Add custom Disallow and Allow directory rules
Include XML sitemap URL directive
One-click copy or direct file download as `robots.txt`
Frequently Asked Questions
It must be placed at the absolute root of your domain: `https://example.com/robots.txt`.