AI Access Rate

AI Access Rate measures what percentage of a site's content is actually accessible to AI crawlers. A site with 500 pages but 200 blocked by robots.txt rules and 100 gated behind JavaScript rendering has an AI access rate of roughly 40% - meaning 60% of its content is invisible to AI systems regardless of how good the content is.

AI access is affected by multiple layers: robots.txt directives (the most explicit), HTTP response codes (a 500 error blocks access), JavaScript rendering requirements (most AI crawlers don't execute JavaScript), CDN bot protection (Cloudflare and similar systems can block AI crawlers), meta tag directives (noindex, noai), and rate limiting (crawlers that are rate-limited may give up and skip pages).

AI access rate is one of the inputs to a brand's AI Discovery Score. A high-quality site with a low AI access rate will have a suppressed score - not because the content is bad, but because AI systems can't reach it. Fixing access issues is often the highest-impact action for improving AI visibility.

The Mentd. platform measures AI access rate via server log analysis and access path testing. It identifies which pages are blocked, why they're blocked, and estimates the impact of fixing each blockage on overall AI visibility.