Docusaurus + AI Readiness
Documentation sites are perfect for AI consumption, but they need an llms.txt to make the structure explicit. w2agent builds it from your docs.
Why Documentation Sites Need AI Readiness
Documentation sites are naturally well-structured — they have clear hierarchies, versioning, and technical content. But without llms.txt, AI models treat your docs like any other website, missing the structure that makes documentation valuable.
No structured index
Docusaurus sidebars define structure for humans, but AI models can't parse sidebar configs.
Version confusion
Multiple documentation versions create duplicate content that AI crawlers may not navigate correctly.
Missing TechArticle schemas
Most Docusaurus themes don't include TechArticle JSON-LD that helps AI classify docs.
Search-only discovery
API references and guides are often only discoverable via Algolia search — invisible to AI crawlers.
What w2agent Generates
w2agent reads your Docusaurus docs structure and generates:
static/llms.txtMaps your documentation hierarchy — categories, guides, API references — into a format AI models understand.
static/llms-full.txtFull markdown content of every documentation page, concatenated for complete AI ingestion.
robots.txt additionsEnsures AI crawlers can access all documentation versions and categories.
TechArticle JSON-LDStructured data for each doc page, including dependencies and proficiency level.
Example: Generated llms.txt for a docs site
# MyProject Documentation > Official documentation for MyProject v2.x ## Getting Started - [Installation](https://docs.myproject.dev/getting-started/install): How to install MyProject - [Quick Start](https://docs.myproject.dev/getting-started/quickstart): Build your first project ## API Reference - [Configuration](https://docs.myproject.dev/api/config): Configuration options - [CLI Commands](https://docs.myproject.dev/api/cli): Available CLI commands
Docusaurus-Specific Checks
| Check | What It Tests |
|---|---|
| llms.txt from docs | Checks if your docs site has an llms.txt that maps to your documentation structure. |
| Structured data for docs | Validates TechArticle and SoftwareApplication schemas on documentation pages. |
Step-by-Step Integration
Run the audit
npx w2agent audit https://your-docs.devGenerate files
npx w2agent generate https://your-docs.devAdd to static/
Copy llms.txt and llms-full.txt to your Docusaurus static/ directory — they'll be served at the site root.
Build and deploy
npm run buildMCP Server for Docusaurus
w2agent can deploy an MCP server that lets AI agents query your documentation site directly. Add a postBuild plugin to call the webhook after each build, keeping the index in sync.
Deploy from your terminal
npx w2agent mcp deploy https://your-docs.devAudit your Docusaurus site now
Get a free AI readiness score and generate the files your site needs.
Start Free AuditRun from your terminal
npx w2agent audit https://example.comnpx w2agent generate https://example.comnpx w2agent integrate https://example.comnpx w2agent mcp deploy https://example.com