theverge.com

Ranked #31 of 136

https://theverge.com
57/ 100
Grade: C
Platform: nextjsPages scanned: 1Duration: 1.7s

Category Breakdown

Bot Accessibility
49
Structured Data
84
Content Quality
89
Agent Discovery
0
Agent Protocols
24
Technical
87

AI Standards

llms.txt
Agent Card
MCP Server

Top Issues

Blocked: ClaudeBot, PerplexityBot, Google-Extended, Bytespider

# Remove or update these rules in robots.txt:
# User-agent: ClaudeBot
# Disallow: /  <-- remove this
# User-agent: PerplexityBot
# Disallow: /  <-- remove this
# User-agent: Google-Extended
# Disallow: /  <-- remove this
# User-agent: Bytespider
# Disallow: /  <-- remove this
+5 pts

No valid XML sitemap found

<!-- Create sitemap.xml at your site root -->
<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://yoursite.com/</loc>
  </url>
</urlset>
+4 pts

No /.well-known/mcp endpoint found

+4 pts

No llms.txt found — LLMs can't discover your site's AI-optimized content

# Create /llms.txt at your site root:
# Your Site Name

> Brief description of your site

## Docs

- [Page Title](https://yoursite.com/page): Description
- [Another Page](https://yoursite.com/other): Description
+4 pts

No discoverable API endpoints (OpenAPI, GraphQL, etc.)

+3 pts

All Checks

Bot Accessibility49/100

Structured Data84/100

Content Quality89/100

Agent Discovery0/100

Agent Protocols24/100

Technical Foundations87/100

Run a fresh audit on theverge.com

Audit now →
AI Readiness: C (57/100) — theverge.com | w2agent