← All tools

🤖 robots.txt Generator

Build a valid robots.txt with rules for search engines and AI crawlers. Includes GPTBot, ClaudeBot, and PerplexityBot.

General settings
Enter paths you don't want any crawler to index. Start each with /
Not supported by Google. Use for server load management.
AI bot access NEW

Control which AI crawlers can access your content for training and citation purposes.

robots.txt output