Is your robots.txt blocking GPTBot, ClaudeBot, or PerplexityBot? Check access for 10 AI crawlers — with the exact robots.txt rule that decides each verdict.
Before ChatGPT, Claude, Perplexity, or Gemini can cite or recommend your site, their crawlers have to be able to read it — and robots.txt is the first gate. Many sites block AI bots by accident: a blanket Disallow added years ago, a CMS default, or a copied robots.txt from another project. This checker fetches your robots.txt and evaluates it against the ten AI crawlers that matter, using real robots.txt semantics: a named user-agent group overrides *, and the longest matching path wins between Allow and Disallow.
If you want to be cited and recommended by AI assistants, allow GPTBot and OAI-SearchBot (OpenAI), ChatGPT-User (ChatGPT live browsing), ClaudeBot and Claude-Web (Anthropic), PerplexityBot, Google-Extended and GoogleOther (Google), and CCBot (Common Crawl, which feeds many AI training sets). Blocking them makes your site invisible to those AI systems.
robots.txt is advisory, not enforcement. Reputable crawlers like GPTBot, ClaudeBot, and Google-Extended honor it, but nothing technically forces compliance. Conversely, a crawler allowed in robots.txt can still be blocked at the CDN or firewall level (Cloudflare, for example, offers one-click AI bot blocking) — so an "allowed" verdict here means robots.txt permits access, not that access is guaranteed.
GPTBot collects content for training OpenAI models. OAI-SearchBot builds the index behind ChatGPT search, which is what gets your site cited in answers. ChatGPT-User fetches pages live when a user asks ChatGPT to visit a link. They respect separate robots.txt rules, so you can allow search and browsing while opting out of training.