Is This Site Blocking AI?
Is your backlink target actually visible to ChatGPT, Claude, and Perplexity?
How AI crawler access shapes your visibility
Assistants like ChatGPT, Claude, Perplexity, and Google's AI Overviews can only cite pages their crawlers are allowed to fetch. This tool reads a domain's robots.txt and checks it against the agents that matter for AI discovery — OpenAI's GPTBot and OAI-SearchBot, Anthropic's ClaudeBot, PerplexityBot, Google-Extended, Amazonbot, and Bytespider — so you can see, agent by agent, whether the page a link points to is reachable or blocked.
Blocking a training crawler (GPTBot, Google-Extended, ClaudeBot) is a content-rights decision; blocking a retrieval crawler (ChatGPT-User, Claude-User) directly removes the page from the sources an assistant can quote. A backlink or PR placement only compounds your AI visibility if the destination is actually crawlable, which is why access is the first thing to verify.
Key takeaways
- Retrieval crawlers (ChatGPT-User, Claude-User) fetch pages at answer time; blocking them removes you from citable sources.
- Training crawlers (GPTBot, Google-Extended, ClaudeBot) shape future models; blocking them is a rights decision, not a visibility one.
- robots.txt is the control surface — a single Disallow can make an otherwise strong page invisible to AI.
- Pair access with an AI Placement Value Score to see whether a reachable domain is also worth earning a link from.
How It Works
Instantly check any site's AI accessibility and placement value
Enter a Domain
Paste any domain you want to check — a potential backlink target, your own site, or a competitor's.
We Analyze Access
We fetch the site's robots.txt and check for restrictions against 40+ AI crawlers and model trainers.
We Score the Domain
We compute an AI Placement Value Score combining organic authority, AI citation access, and training influence.
Get Actionable Results
See which AI bots are blocked, the domain's placement value, and recommendations to maximize AI visibility.
Why This Matters for SEO & PR
AI visibility is the new battleground for brand discovery
Evaluate Backlink Quality
Before pursuing a backlink, check if the target site is even visible to AI assistants. A site that blocks ChatGPT and Claude won't pass AI discovery value, no matter how authoritative.
Maximize AI Discovery
AI assistants are increasingly how consumers research products and services. If your content or your earned media placements are blocked, you're missing out on a growing channel.
Use This Data in Your Earned Media Workflow
Want to see how a placement on this publisher influences your site, or your client's? Start a free Spyglasses trial to score any domain, track AI visibility trends, and make data-driven placement decisions.
Frequently Asked Questions
What does this tool check?
This tool analyzes a website's robots.txt file to determine if AI crawlers and chatbots are blocked from accessing the content. It checks for restrictions against AI visitors like ChatGPT, Claude, and Perplexity, as well as AI model trainers like GPTBot and Google-Extended.
Why does AI access matter for SEO and PR?
AI assistants like ChatGPT, Claude, and Perplexity are increasingly used by consumers to research products and services. If your content is blocked by robots.txt, these AI systems cannot cite your website in their responses, potentially reducing your brand's visibility and the value of backlinks pointing to your site.
What's the difference between red, yellow, and green status?
Red status means AI visitors (bots that cite content in real-time like ChatGPT and Claude) are blocked - this has immediate visibility impact. Yellow status means AI model trainers (bots that collect training data like GPTBot) are blocked - this affects future AI model knowledge. Green status means no significant AI restrictions.
How does this affect the value of my backlinks?
If a site blocks AI crawlers, any backlink you earn from that site may have reduced value in AI-driven search and recommendations. AI assistants won't be able to discover or cite content from sites with restrictive robots.txt policies, limiting the reach of your earned media placements.
Should I block AI crawlers on my site?
It depends on your goals. Blocking AI model trainers can prevent your content from being used to train future AI models, but blocking AI visitors prevents your content from being cited in AI-generated responses. Most brands benefit from allowing AI visitors to maximize visibility while potentially restricting model trainers.
How do I fix robots.txt issues?
To allow AI crawlers, you need to modify your robots.txt file to remove or adjust User-agent rules that block them. This typically involves allowing user agents like GPTBot, ChatGPT-User, Claude-Web, and PerplexityBot. See our detailed guide for specific instructions.
What if no robots.txt is found?
If a website has no robots.txt file, all crawlers (including AI bots) have unrestricted access. This is actually good for AI visibility - it means your content can be discovered and cited by AI assistants. However, you may want to create a robots.txt to have more control over which bots can access your site.
Can I track when AI bots visit my site?
Yes! Spyglasses AI Traffic Analytics can detect and log visits from AI crawlers in real-time. You can see which AI bots are visiting your site, how often, and what content they're accessing. You can even use our traffic control features to selectively block or allow specific bots without modifying your robots.txt.
Checking whether an outlet blocks AI? AIPVS turns that check into a full publisher score.
AI Placement Value Score →Score the specific article too, landed or still a pitch.
AI Placement Quality Score →See which publishers AI actually cites in your market.
Citation Intelligence →PR teams: this check is step one of the whole earned-media toolkit.
For PR & Comms →