Token efficiency (HTML-to-text ratio): what the check tests and how to fix it

Weight 1. Averages visible-text length over raw HTML length across crawled pages. 10% or more passes, 25% scores 100. Thresholds are the audit's own.

Content Quality · Weight 1 in the w2agent Score

How common is it?

18% of 1223 measured sites where this check applied pass.

Latest audit of each site under rubric v3.1. Sites where the check did not apply or could not be measured (for example, no llms.txt for the llms.txt checks) are left out. A partial result does not count as a pass.

What this check tests

For each crawled page (the homepage plus up to nine more) the audit divides two character counts:

  • Text length: the visible body text after scripts, styles, noscript and template content are removed and whitespace is collapsed. Navigation, header and footer text stay in.
  • HTML length: the length of the raw HTML string the server returned, including the head, inline CSS, inline scripts, JSON data blobs and markup.

A page with an empty HTML string counts as a ratio of 0. The check then takes the plain average of the page ratios, so a small page weighs as much as a large one, and maps it:

Average ratioScoreStatus
25% or more100Pass
15% to under 25%80Pass
10% to under 15%60Pass
5% to under 10%30Partial
Under 5%0Fail

The message gives the average to one decimal place. The pass line is 10%, below the 15% where the score first reaches 80. The ratio counts characters, not tokens, and the thresholds are this audit's own choice. No standard or vendor document defines a good value.

Why it matters to agents

Fetching tools do pay for page size. Anthropic's web fetch documentation says fetching full web pages can quickly consume tokens, and that its max_content_tokens parameter truncates fetched content that exceeds the limit. Whether a given tool strips markup before applying such a limit is not stated in these sources, so the effect of a low ratio on any one tool is unknown. Cloudflare, in its own announcement, reports one example: its blog post took 16,180 tokens as HTML and 3,150 as markdown, an 80% reduction. That is one page measured by the vendor, not a typical rate.

Google's agent-friendly site guide notes that agents may read a site through screenshots, the HTML and the accessibility tree. A low ratio does not say how any one agent behaves. It says your pages carry a lot of non-text weight.

A very low ratio can also mean the opposite problem: a page with little server-rendered text, such as a client-rendered shell. That case is covered by no JavaScript dependency.

How to fix it

  1. Find the heavy pages. View source and look for large inline <script> blocks, serialized app state, inline SVG sprites, base64 images and repeated utility classes.
  2. Move scripts, styles and data to external files, and load data on demand rather than embedding it in every page.
  3. Remove duplicated markup, such as menus rendered twice for desktop and mobile, where you can.
  4. Do not cut real content to raise the number. A page that is mostly text and still scores low is fine to leave.
  5. Offer a lighter representation for agents that ask for it: markdown alternate links and content negotiation cover that.

Related: content length uses word count rather than a ratio, and semantic HTML covers the structure of the markup that remains.

Examples from measured sites

Up to 10 of the highest-scoring sites in each group, from recent audits. Failing includes partial results.

Passing

Failing or partial

Sources

Verified against these sources on

Does your site pass this check?

Check your own site →