Docusaurus + AI Readiness
Documentation sites are perfect for AI consumption, but they need an llms.txt to make the structure explicit. w2agent builds it from your docs.
Step-by-step guides
- How to add llms.txt to a Docusaurus site (llms.txt)
Why Documentation Sites Need AI Readiness
Documentation sites are naturally well-structured — they have clear hierarchies, versioning, and technical content. But without llms.txt, AI models treat your docs like any other website, missing the structure that makes documentation valuable.
No structured index
Docusaurus sidebars define structure for humans, but AI models can't parse sidebar configs.
Version confusion
Multiple documentation versions create duplicate content that AI crawlers may not navigate correctly.
Missing TechArticle schemas
Most Docusaurus themes don't include TechArticle JSON-LD that helps AI classify docs.
Search-only discovery
API references and guides are often only discoverable via Algolia search — invisible to AI crawlers.
What w2agent Generates
w2agent reads your Docusaurus docs structure and generates:
static/llms.txtMaps your documentation hierarchy — categories, guides, API references — into a format AI models understand.
static/llms-full.txtFull markdown content of every documentation page, concatenated for complete AI ingestion.
robots.txt additionsEnsures AI crawlers can access all documentation versions and categories.
TechArticle JSON-LDStructured data for each doc page, including dependencies and proficiency level.
Example: Generated llms.txt for a docs site
# MyProject Documentation > Official documentation for MyProject v2.x ## Getting Started - [Installation](https://docs.myproject.dev/getting-started/install): How to install MyProject - [Quick Start](https://docs.myproject.dev/getting-started/quickstart): Build your first project ## API Reference - [Configuration](https://docs.myproject.dev/api/config): Configuration options - [CLI Commands](https://docs.myproject.dev/api/cli): Available CLI commands
Docusaurus-Specific Checks
| Check | What It Tests |
|---|---|
| llms.txt from docs | Checks if your docs site has an llms.txt that maps to your documentation structure. |
| Structured data for docs | Validates TechArticle and SoftwareApplication schemas on documentation pages. |
Step-by-Step Integration
CLI coming soon — run the web audit
Generate files
Run the web audit, open the Manual Fixes tab, click Generate, and download the files for your platform.
Agent for Docusaurus
w2agent can agentize your documentation site — deploying an agent so AI agents can query it directly. Add a postBuild plugin to call the webhook after each build, keeping the index in sync.
CLI coming soon — run the web audit
Score your Docusaurus site now
Get your free w2agent score and generate the files your site needs.
Get Your ScoreCLI coming soon — run the web audit