llms.txt is not a standard. Publish one anyway.

There is a version of this post that tells you llms.txt is essential and that everyone reading it is behind. That version would sell more Reviews. It would also be wrong.

What llms.txt actually is

A markdown file at your domain root that says what your site is and points at the pages that matter. It was proposed by Jeremy Howard↗(opens in a new tab) in 2024 as a way to give language models a curated map instead of making them guess from a sitemap.

The idea is sensible. The adoption is the question.

The honest status

No major AI company has publicly committed to reading llms.txt as a ranking or retrieval input. Some tools consume it. Some crawlers appear to fetch it. Nobody has published a specification saying "we use this and here is how".

Anyone telling you otherwise is either guessing or selling.

So why publish one

Three reasons, in descending order of confidence.

It costs almost nothing. Fifteen minutes, one file, no risk of breaking anything. The expected value only has to be slightly above zero for that trade to be worth making.

It forces a useful exercise. Writing an llms.txt means deciding, in one paragraph, what your business does and which ten pages matter most. Most businesses have never written that down. The file is often less valuable than the thinking it requires.

The downside is zero and the upside is asymmetric. If adoption happens, you are already there. If it does not, you wasted a quarter of an hour and gained a clear summary of your own site.

What a good one looks like

Not a sitemap. A sitemap lists everything; an llms.txt curates. One paragraph on what you do, then linked sections for the pages you would want quoted, each with a sentence saying why it exists.

We have a longer guide and a generator that writes one from your existing structure. Both are free and always will be.

The argument against

The strongest case for skipping it: llms.txt duplicates information that already exists in your sitemap and your structured data, and maintaining a third source of truth means a third thing to go stale. If your site changes weekly and nobody owns the file, a wrong llms.txt is worse than none.

That is a fair objection. My answer is that it argues for keeping the file short rather than for not having one — but if you know you will never update it, the objection stands.