Astro

Astro + AI Readiness

Astro's static-first architecture is ideal for AI crawlers, but you still need llms.txt and proper configuration. Here's how.

Why Astro Is Great for AI (With a Few Gaps)

Astro's static HTML output is exactly what AI crawlers want — fully rendered pages with no JavaScript dependency. But most Astro sites still miss key files AI agents look for.

No llms.txt

AI models need a structured index of your site. Astro doesn't generate llms.txt — your content is invisible to LLMs.

Missing agent-card.json

The .well-known/agent-card.json endpoint enables AI agent discovery — not included by default.

robots.txt may block AI

Default or third-party robots.txt configs sometimes block GPTBot, ClaudeBot, and other AI crawlers.

No structured data

Static Astro pages often lack JSON-LD markup that helps AI understand your content semantically.

What w2agent Generates

Static files first (Astro's strength), with optional SSR endpoints:

public/llms.txt

Static file — zero overhead, works on any host. This is the primary serving method.

public/.well-known/agent-card.json

Static agent discovery file — served directly from public/.

w2agent-integration.mjs

Astro integration plugin that regenerates files at astro:build:done.

src/pages/llms.txt.ts

Optional SSR API endpoint for dynamic content (when using output: 'server').

Example: Astro build integration

// astro.config.mjs
import w2agent from "./w2agent-integration.mjs";

export default defineConfig({
  integrations: [w2agent()],
});

// Regenerates llms.txt at build time via astro:build:done hook

Astro-Specific Checks

CheckWhat It Tests
Sitemap integrationChecks if @astrojs/sitemap is installed so AI crawlers can discover all your pages.
robots.txt AI bot accessVerifies robots.txt exists and doesn't block AI crawlers like GPTBot or ClaudeBot.
Rendering mode detectionIdentifies static vs SSR mode — static is ideal for AI crawlers, SSR needs proper cache headers.

Step-by-Step Integration

1

Run the audit

npx w2agent audit https://your-astro-site.com
2

Generate files

npx w2agent generate https://your-astro-site.com
3

Add to your site

Copy static files to public/ — they work immediately with zero config. Optionally add SSR endpoints and the build integration for auto-regeneration.

4

Deploy

Deploy to any static host (Vercel, Netlify, Cloudflare Pages). The build integration regenerates content at astro:build:done.

Audit your Astro site now

Get a free AI readiness score and generate the files your site needs.

Start Free Audit

Run from your terminal

npx w2agent audit https://example.com
npx w2agent generate https://example.com
npx w2agent integrate https://example.com
npx w2agent mcp deploy https://example.com
Astro AI Readiness | w2agent