SEMPITE Research · Ecosystem Audit · August 2026
We Audited 1,563 llms.txt Files. One in Six Is Already Broken.
llms.txt is the web’s new instruction manual for AI — a plain-text file that tells language models what a site is and where its important pages live. Adoption is celebrated. Maintenance, it turns out, is not. We fetched every llms.txt file listed in the ecosystem’s own public directories and then validated 19,039 of the links inside them. The format designed to make sites legible to AI is quietly rotting — and almost nobody is checking.
File-level health: 1 in 6 doesn’t work at all
Of 1,563 llms.txt files listed across the ecosystem’s public directories (llmstxt.site, directory.llmstxt.cloud, llmstxthub.com), here is what actually resolves:
The two failure modes nobody notices: 32 sites return a 200 OK for llms.txt — but serve their homepage HTML instead of the text file (a single-page-app fallback). The file looks alive in every monitoring tool and is useless to a language model. And 54 sites return 403 Forbidden to crawlers: they published a file whose entire purpose is to be read by bots, then pointed their bot protection at it.
Link-level health: what’s inside the files that do work
We validated 19,039 links from 1,147 healthy files — every link (up to 400) for roughly a hundred notable documentation sites, and a random sample of 15 links per site for the long tail:
A further 5.2% of links only resolve after a redirect — the URL the file hands to an AI is not the URL that exists — and 19 links landed on a login page: technically a 200, practically a locked door. In total, roughly 1 link in 16 inside the ecosystem’s llms.txt files does not deliver the content it promises. 209 of 1,147 sites (18.2%) link to at least one dead page from the file they wrote specifically to guide AI.
A worked example: Stripe
Stripe’s developer documentation is widely considered the gold standard, which is exactly why it makes the point. In July, stripe.com/llms.txt directed AI crawlers to a “complete documentation” file at stripe.com/llms-full.txt — which returned a 404. We reported it to Stripe on July 12.
- July 11 — live probe finds llms.txt pointing AI crawlers to llms-full.txt, which 404s.
- July 12 — we report the broken pointer to Stripe.
- August 5 — the pointer has been removed from llms.txt, but llms-full.txt still 404s on both stripe.com and docs.stripe.com. And a fresh validation of all 768 links across Stripe’s two llms.txt files finds 8 dead documentation pages (Treasury, Issuing compliance, and reporting docs) plus 2 links that redirect AI crawlers to a dashboard login wall.
If the company with the best documentation culture on the internet ships dead ends in its AI manifest, the takeaway isn’t that Stripe is careless — it’s that nobody’s build pipeline treats llms.txt as production. Docs move, products sunset, handles change — and the file written for machines is updated by hand, or not at all.
Why this matters
An AI assistant that follows a dead link doesn’t file a bug report. It silently falls back to whatever it already believes about you — stale training data, a competitor’s comparison page, a Reddit thread. Every broken link in an llms.txt file is a moment where you handed the microphone to someone else. And unlike a human 404, there is no analytics event for an AI hitting a dead end. The failure is invisible unless you go looking.
The fix is unglamorous: treat llms.txt like production infrastructure. Validate its links in CI the way you validate your sitemap. Alert on 404s. Exempt it — explicitly — from your bot protection. The sites that do this will be the ones AI systems can actually read; right now, that bar is surprisingly low.
Want to know what AI assistants find — or fail to find — when they read your site?
Run a free AI visibility check