What this check tests
The audit requests the page you submitted once with the Google-Agent string, the third and last of its scored reach requests. Pass and fail follow the rule shared with ChatGPT-User and Perplexity-User: a 2xx status with a non-empty body passes at 100, any other status, an empty body, a timeout or a network error fails, and a missing record or an exhausted time budget is unmeasured and not scored. The ChatGPT-User page has the full table and explains how reach checks differ from AI agent access and edge agent access.
What is specific to this one:
- The header is Google's desktop agent string:
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko; compatible; Google-Agent; +https://developers.google.com/crawling/docs/crawlers-fetchers/google-agent) Chrome/W.X.Y.Z Safari/537.36. Google also lists a mobile variant, which the audit does not send. - The three requests share a time budget, the smaller of three timeouts and 15 seconds, checked before each fetch. With the standard profile's 10-second timeout, two slow responses ahead of it can exhaust the budget, and then this check reports unmeasured rather than failed. Perplexity-User and ChatGPT-User cannot be skipped that way.
- Google-Agent has no robots.txt token. It is not among the tokens AI agent access tests, so robots.txt has no part in this result at all.
Why it matters to agents
Google documents Google-Agent as used by agents hosted on Google infrastructure to navigate the web and perform actions on a user's request. It sits on Google's list of user-triggered fetchers, and Google says that, because a user requested the fetch, these fetchers generally ignore robots.txt rules. The list publishes the IP ranges Google-Agent uses, in a file called user-triggered-agents.json, and gives reverse DNS masks for verifying fetchers.
Google also says it is experimenting with Web Bot Auth, an IETF draft for signing requests, and that its implementation is experimental: only some Google-Agent requests are signed, and Google recommends continuing to rely on IP addresses, reverse DNS and user-agent strings. That is why this check does not look for signatures.
A pass here means your server does not reject the Google-Agent string from the audit's address. Because the real agent comes from Google's ranges, a firewall that verifies those ranges can treat the real agent differently. Nothing fetched for this page says what a Google-Agent visit does for your visibility in Search, and Google's own guidance on AI features says no extra technical requirements apply for appearing in AI Overviews or AI Mode. This check is about task agents reaching a page, not about ranking.
How to fix it
- Read the message. A status such as 403 or 429 means a rule is reacting to the string. A timeout points to slow responses.
- Check your CDN or WAF for bot rules that cover "AI agents" or all unrecognised automation. Many block by user-agent substring.
- If you want to allow Google-Agent, use Google's published ranges or reverse DNS together with the user-agent, which Google says to keep relying on, or a product that supports Web Bot Auth verification. Some services provide a built-in way to allow Google-Agent; use your provider's documentation for the exact steps.
- If the result is unmeasured, the earlier fetches in the queue were slow. Improve response time and re-run.
- If you intend to block it, the failing result is the intended outcome.
If you operate a bot yourself and want others to verify it, agent operator signing covers publishing a key directory. Background: AI crawler blocking mistakes.