What this check tests
This is a tracked check: it does not change your score. It is about the operator of a bot, not about whether agents can use your site, so it only means something if your organization runs a crawler or agent that signs its requests.
The audit requests /.well-known/http-message-signatures-directory. The crawl accepts the response only if the status is 200, the media type is exactly application/http-message-signatures-directory+json, and the body differs from what your server returns for a random path under /.well-known/ (so a catch-all page is not mistaken for a file). Then:
- Skip: nothing accepted. Most sites get this, and it is not a finding.
- Fail: the body is not a JSON object with a
keysarray containing an object withktyOKP,crvEd25519and a stringx. - Fail: it has such a key, but no
signature_agentmember, or one whose origin differs from the audited origin. - Pass: both hold.
The audit does not verify any signature, the signature on the directory response, the key validity dates, or that your bot's traffic actually uses these keys.
One requirement here is the audit's own. The IETF Web Bot Auth drafts checked for this page, including the current working group draft (draft-ietf-webbotauth-httpsig-protocol-00) and the earlier directory drafts 00 to 05, define the directory as a plain JSON Web Key Set with a keys array. The origin of the bot is carried in the Signature-Agent request header, not in a signature_agent JSON member, and none of the drafts defines such a member. A directory that follows the draft exactly will therefore fail this check on the second condition.
Why it matters to agents
Web Bot Auth lets an automated client prove who it is. The client signs requests with HTTP Message Signatures (RFC 9421) and adds a Signature-Agent header naming where its public keys are published. A site that wants to tell a real crawler from an impersonator fetches the directory and verifies the signature. The draft registers the well-known path and the media type for that directory, and requires the directory to be served over HTTPS.
Cloudflare documents the same approach for its verified bots: you publish the directory, sign each request, and register the bot with Cloudflare. In its documentation the directory is a standard JSON Web Key Set, and Cloudflare reads only kty, crv and x of an Ed25519 key.
This matters to the site that receives the traffic and decides whether to trust a bot. It tells you nothing about how well a given website serves agents, which is why the check is tracked rather than scored. Web Bot Auth is still an Internet-Draft, so details can change.
How to fix it
Only if you operate a signing agent:
- Generate an Ed25519 key pair and keep the private key out of the web root.
- Publish the public key as a JWKS at
https://bot.example.com/.well-known/http-message-signatures-directory, withContent-Type: application/http-message-signatures-directory+json:
{
"keys": [{ "kty": "OKP", "crv": "Ed25519", "x": "<base64url public key>" }],
"signature_agent": "https://bot.example.com"
}
- Sign your requests per the draft and RFC 9421, and send
Signature-Agentwith the same origin.
The signature_agent member is there only because this audit expects it. Cloudflare's documentation says it ignores key parameters other than kty, crv and x; none of the sources checked for this page says whether a verifier tolerates an extra top-level member, so check your verifier before adding it. Serve the file as a real file, not through a catch-all route.