Free Tool — No Signup Required
Robots.txt AI Bot Checker
Check which AI crawlers (GPTBot, ClaudeBot, PerplexityBot...) are blocked or allowed by your robots.txt file.
How It Works
1
Enter your domain
Type any domain name to check its robots.txt configuration for AI bots.
2
We analyze robots.txt
We parse the robots.txt file and check rules for 14 AI crawlers including GPTBot, ClaudeBot, and more.
3
Get your report
See which AI bots can crawl your site and which are blocked. Plus, check if you have an llms.txt file.
Why AI Bot Access Matters
AI search engines like ChatGPT, Perplexity, and Google Gemini use web crawlers to index content. If your robots.txt blocks these bots, your site may not appear in AI-generated answers.
Blocking AI bots means missing out on a growing source of traffic and brand visibility. Check your configuration now to make sure you're not invisible to AI.