Free Tool
Robots.txt Generator
Generate a robots.txt file instantly. Use ready-made templates for WordPress, e-commerce, and AI crawler blocking — or build custom rules. One-click download.
Allow All
Allow all search engine crawlers to access your entire website.
Block AI Crawlers
Allow search engines but block AI training bots like GPTBot, Claude-Web, and PerplexityBot.
WordPress
Block WordPress admin areas from being indexed while allowing the rest of your site.
E-Commerce
Block cart, checkout, account, and search pages. Ideal for WooCommerce and Shopify stores.
Block Admin Areas
Block admin, login, dashboard, and private pages from search engine crawlers.
User-agent: * Allow: / Sitemap: https://example.com/sitemap.xml
Ready to Take This Further?
ModernWebSEO combines technical SEO, web design, and digital marketing. Get a free consultation and see what we can build together.