llms.txt Doesn't Work. Here's the Data.

Publishing an llms.txt file does almost nothing for your AI visibility. The AI assistants you are trying to reach do not read it. Ahrefs analyzed the server logs of 137,000 domains and found that 97% of llms.txt files got zero fetches in a month, and that the fetches that did happen came mostly from SEO tools, not from ChatGPT or Perplexity (Ahrefs, June 2026). Google's own documentation says machine-readable files like llms.txt are not needed to appear in AI search. So before you spend an afternoon writing one, know what you are actually buying: a file that is cheap to keep and safe to ignore. The work that moves AI visibility is somewhere else entirely, and this post shows you where.

What llms.txt Is Supposed to Do

llms.txt is a single markdown file placed at the root of your site. Jeremy Howard, of Answer.AI and fast.ai, proposed it in 2024 as a curated index that points AI models at the pages that matter, in a clean format they can read without wading through your navigation and scripts.

Two things it is not. It is not the practice of publishing markdown copies of your pages, which is a separate idea people often conflate with it. And despite the name, it is not a robots.txt-style directive. robots.txt tells crawlers what they may fetch. llms.txt controls nothing, blocks nothing, and grants nothing. It is a suggestion sitting on your server, waiting for a reader that, as the data shows, never arrives.

The Data: 97% of These Files Are Never Read

The clearest evidence comes from Ahrefs, which analyzed 137,000 domains using its Web Analytics and Bot Analytics data in May 2026. The findings are hard to argue with.

Finding	Number
Domains publishing an llms.txt file	28%
Those files that got zero fetches in a month	97%
Fetches to llms.txt files that came from bots (not humans)	96%
Requests from AI bots for llms.txt files that don't exist	0

Read that last row again. AI crawlers never went looking for the file. When it was absent, they did not request it and move on. They simply never asked. Of the small share of files that were fetched at all, most of the traffic came from SEO audit tools, tech profilers, and llms.txt checkers, which is the industry studying itself. Ahrefs notes its audience skews technical, so 28% adoption is an upper bound, not a floor.

Even Google Says You Don't Need One

The strongest signal is that the platforms themselves have said it out loud. In a 2026 guide on optimizing for AI features, Google included a section literally titled "mythbusting" that told site owners machine-readable files like llms.txt are not required to show up in generative AI search.

Google's John Mueller was even blunter when pressed on the contradiction between that guidance and a Chrome experiment that checks for the file. He said llms.txt is "not done for search" and described it as a temporary crutch that might save some tokens for AI coding tools parsing developer docs. That is a narrow, developer-tooling use case. It is not a discovery channel, and it is not how ChatGPT decides whether to recommend your brand. When the company that runs the largest AI search surface tells you a file is not needed, believe it.

Why the Myth Spreads Anyway

If the file does nothing, why is your feed full of "how to create an llms.txt" guides? Because it is easy to write about and easy to ship. A tactic that takes ten minutes and produces a visible artifact feels like progress. It photographs well in a checklist.

There is also a vendor incentive. Several AEO tools ship llms.txt generators, so the file gets bundled into "AI optimization" packages as a feature. Ahrefs found that GEO and AEO tools studying the file were themselves a measurable slice of the fetches. The industry is fetching its own invention and citing the activity as proof of relevance. None of this makes the file reach an actual AI assistant. It makes the file look busy, which is a different thing from making it useful.

When Keeping an llms.txt Is Still Fine

Here is the honest part. Having an llms.txt file is not harmful, and there are reasons a careful team might keep one. Lectern publishes its own. That is not a contradiction, it is a bet on cheap optionality.

The case for keeping one comes down to three things. It costs almost nothing to maintain. If a coding agent or a niche tool ever does parse it, you lose nothing by having it ready. And standards sometimes win adoption after the fact, so a low-cost hedge against a future where the file matters is defensible. What you should not do is treat it as a visibility strategy, put it ahead of real work, or report it to a client as an AI-visibility deliverable. Keep it as hygiene if you like. Do not sell it as a result.

What Actually Moves AI Visibility

The reason AI ignores your llms.txt is the same reason it cites some brands and not others: models answer commercial questions by running a search and summarizing what they find. So the levers are the ones that shape what gets found and quoted.

Instead of llms.txt	Do this
Curating a file no crawler reads	Ship crawlable, server-rendered HTML so models can actually parse your pages
Hoping a file explains your product	Answer the buyer's question in the first sentence, then structure with headings, tables, and FAQs
A root-level index	Earn third-party mentions on the sources models cite (Reddit, YouTube, review sites, industry roundups)

The pattern is consistent across every serious AEO playbook. Make your critical content readable without JavaScript. Write answer-first pages that a model can lift a clean passage from. And get mentioned across the sources AI synthesizes, because being cited often matters more than ranking first. If you want the full version of this, start with our guide to what AEO is and then run an AI visibility audit to see where you actually stand.

The Bottom Line

llms.txt is not a scam and it is not a scandal. It is a low-cost file that a small, developer-focused audience occasionally uses and that the AI assistants driving your visibility do not read. The data is clear: 97% of these files get zero fetches, and Google says they are not needed. Keep one if it makes you feel tidy. Just do not confuse tidiness with traffic. The brands winning in AI answers are not the ones with the best-formatted root file. They are the ones with crawlable pages, answer-first content, and mentions across the web that models trust. That is where your afternoon should go.

---

Frequently Asked Questions About llms.txt

Does ChatGPT read llms.txt?

No evidence suggests it does. Ahrefs analyzed 137,000 domains and found 97% of llms.txt files received zero fetches in a month, with most of the small remainder coming from SEO tools rather than AI assistants. AI crawlers did not even request the file when it was absent. Treat it as unread by the major assistants.

Is llms.txt the same as robots.txt?

No. robots.txt is a directive that tells crawlers what they are allowed to fetch, and crawlers respect it. llms.txt controls nothing and blocks nothing. It is a proposed index file that suggests which pages matter, but nothing enforces or requires that a model use it.

Should I delete my llms.txt file?

You do not need to. The file is cheap to keep and does no harm. The mistake is treating it as an AI-visibility strategy or a client deliverable. Keep it as low-cost optionality if you want, but do not prioritize it over crawlable HTML, answer-first content, and third-party mentions.

Why do so many guides recommend llms.txt?

Because it is easy to write about and quick to ship, which makes it feel like progress. Some AEO tools also bundle llms.txt generators as a feature, so the tactic spreads through vendor packages. Ease of publishing is not the same as effectiveness.

What should I do instead of llms.txt?

Focus on the three things models actually use: crawlable, server-rendered HTML so your pages can be parsed; answer-first content structured with headings, tables, and FAQ blocks that are easy to quote; and third-party mentions on the sources AI cites. Those shape what gets found and recommended.

Does Google recommend llms.txt?

The opposite. Google's 2026 guidance on optimizing for AI features includes a mythbusting section stating that machine-readable files like llms.txt are not needed to appear in generative AI search. Google's John Mueller called it a temporary crutch for AI coding tools, not a search feature.

---

Lectern helps growth-stage brands get recommended by AI assistants. We measure how you show up across ChatGPT, Gemini, Perplexity, Claude, Grok, and Meta AI, benchmark you against competitors, and close the gap with content and publishing systems built over years in traditional media. See how it works.