Is llms.txt worth it in 2026? What the data actually says

For two years, llms.txt has been sold as the robots.txt of the AI era - a simple file that tells language models how to read your site so they cite you more. In 2026 the data arrived, and it is not kind to that pitch. Here is what llms.txt does, what it doesn't, and where your effort actually belongs.

The idea is appealing. Publish a plain-text file at your domain that points models to your best content, and you nudge your way into more AI answers. Plenty of "AEO checklists" still list "add llms.txt" near the top. The trouble is that the engines those checklists are optimizing for mostly ignore it.

What the data shows

The most thorough look came from Ahrefs, which analyzed 137,210 domains. Two findings stand out. First, adoption is real but modest - around 28% of the domains studied had published an llms.txt file. Second, and more damning: 97% of those files received zero requests in the month studied. The file existed; nothing came to read it.

Of the small fraction of files that did get traffic, the biggest category was not answer engines at all - it was agentic coding tools. Ahrefs' own conclusion was blunt: if your goal is showing up in ChatGPT, Perplexity or AI Overviews, an llms.txt file is largely decoration.

Google has said the quiet part out loud

It is not just crawl data. Google's Search team has repeatedly said that Search does not use or endorse llms.txt. One Google representative likened it to the long-deprecated keywords meta tag - a signal you can declare that the system simply does not consult. When the largest search engine tells you a file has no bearing on how it reads you, that is worth believing.

None of the major answer engines have committed to reading llms.txt as a ranking or citation input. A standard only works if the systems it is meant for adopt it, and so far they haven't.

Where llms.txt genuinely helps

This is not "llms.txt is useless." It is "llms.txt is useful for a different job than the one it is marketed for." Its real, working use is helping coding and documentation agents - IDE assistants and dev tools - locate and read a site's technical docs efficiently. If you run a developer-facing documentation site, an llms.txt that indexes your docs can genuinely make those agents faster and more accurate.

That is a legitimate, narrow use. It is just not the same thing as "get cited by ChatGPT when a buyer asks for the best tool in your category." Conflating the two is where the wasted effort comes from.

"If your goal is showing up in ChatGPT, Perplexity or AI Overviews, an llms.txt file is largely decoration."

What to do instead

If AI visibility is the goal, spend your time on the inputs engines demonstrably use:

1. Crawlable, well-structured pages. The oldest advice is still the load-bearing advice. A model can only cite what it can read cleanly.
2. Clear, self-contained facts. State your category, audience, pricing and differentiators plainly, so a model can lift them without guessing.
3. Accurate schema markup. Structured data that describes what a page is and who you are - the kind engines actually parse - earns its keep.
4. Corroboration. The same accurate description of you, echoed across the third-party sources models trust, does more than any file on your own domain.
5. Measurement. Track whether you are actually named and cited. That is the only signal that tells you what is working, rather than what a checklist promised.

What llms.txt was actually designed to do

It helps to go back to the beginning, because the original intent was never "rank in ChatGPT." Jeremy Howard, co-founder of Answer.AI, proposed llms.txt in September 2024 to solve a narrow, real problem: language model context windows are small, and a modern web page is mostly not content. Navigation, cookie banners, ads, and JavaScript all have to be stripped before a model can read the few paragraphs that matter, and that conversion is lossy and expensive.

llms.txt was the fix. Publish a single clean Markdown file at your domain root that curates your most important pages and links to plain-text versions of them. A tool can point its context window at that one file instead of scraping and cleaning your HTML. The spec even defines a companion llms-full.txt that inlines the full text of your docs into one document. Read that way, llms.txt is a convenience for a program that already decided to read you and knows your file exists. It was never designed as a discovery signal that makes an engine choose to cite you. That distinction is the whole story, and most of the marketing around the file quietly erases it.

Why answer engines never picked it up

A file-based standard only works if the systems it targets agree to read it, and answer engines had good reasons not to. Three, specifically:

• Trust. A model that ranks and cites sources cannot lean on a self-declared file that a site author controls end to end. llms.txt is an unverified claim about what matters on your domain. The moment it influenced citations, every SEO shop on earth would stuff it, which is exactly why Google likened it to the keywords meta tag it abandoned for the same reason.
• Redundancy. Engines like Google already crawl and render your actual pages at scale. They do not need a hand-curated map to find your content, and they trust what they rendered themselves over what you asserted in a text file.
• No adoption commitment. None of the major answer engines committed to reading llms.txt as a ranking or citation input, so there was never a reason for it to gain traction on the retrieval side.

The crawl data makes the indifference concrete. Beyond the Ahrefs finding that 97% of llms.txt files were never fetched, across large samples of AI-bot traffic, only a vanishingly small share of requests ever target llms.txt at all. The bots that matter for citations are, overwhelmingly, walking straight past the file.

The coding-agent use case, explained properly

So where does the file earn its keep? Inside developer tools, and the mechanism is worth understanding because it is genuinely useful in the right context.

When you point an IDE assistant at a documentation site, it has to gather relevant docs into a limited context window. Cursor, Claude Code, Windsurf, GitHub Copilot, Cline, and Aider have all added support for looking up /llms.txt and /llms-full.txt when they are working against a site's docs. Serving curated Markdown instead of full HTML has been reported to cut token usage dramatically, which means faster, cheaper, and more accurate agent behavior. In practice it is the difference between an assistant generating a working API call and one hallucinating an endpoint that never existed.

This is why the companies shipping llms.txt are almost all API and developer-platform companies - the ones whose users are actively building against their docs with AI assistants. If that describes you, the file does a real job:

• Point it at your API reference, quickstart, and authentication docs - the pages an agent needs to write correct integration code.
• Keep it curated. A short list of your canonical pages beats an autogenerated dump of every URL you have.
• Consider an llms-full.txt for your core reference so an agent can load it in one fetch.
• Keep it current. A stale file that points to renamed or deleted pages is worse than none, because it sends the agent confidently to the wrong place.

Notice that none of this touches whether a buyer's "best tool for X" question surfaces your brand in ChatGPT. It is a docs-quality feature for people already inside your ecosystem, not a top-of-funnel visibility play.

Myth versus reality

It is worth naming the specific claims that circulate, because most llms.txt confusion comes from one true statement being stretched into a false one. The myth is that llms.txt is the robots.txt of AI, a standard the engines have agreed to honor; the reality is that no major answer engine has committed to it as a ranking or citation input. The myth is that publishing one earns you AI citations; the reality is that 97% of published files are never fetched by anything, so most produce no measurable signal at all. The myth is that ignoring it means falling behind; the reality is that roughly 28% of studied sites already publish one and adoption has not moved their answer-engine visibility, so there is no first-mover advantage to miss. The one claim that survives contact with the data is the narrow one: llms.txt helps coding and documentation agents read your docs efficiently. Keep that claim, drop the rest, and you have calibrated your expectations correctly.

The honest verdict

Adding llms.txt is cheap, so if you run a docs site and want to help coding agents, go ahead - it is low-cost and occasionally useful. But do not add it expecting more AI citations, and do not let it crowd out the work that moves the needle. In 2026, llms.txt is a small convenience for developer tools, not a visibility strategy. The visibility strategy is still clear facts, clean structure, real corroboration, and honest measurement.

Related: Schema markup for AEO - which structured data helps AI cite you →