Skip to main content
Contact

llms.txt: what it does, who reads it, and whether you need one

llms.txt is a markdown file at the root of a website. It tells AI tools, and the large language models behind them, what the site is and where its main pages are. Jeremy Howard proposed it in September 2024 and revised the proposal in August 2026. Google Search does not require it, and the evidence that AI search uses it is thin.

We publish one, and a markdown copy of our content pages. This post covers what each does, what the evidence says, and what we would tell a client before they pay for either.

What llms.txt is

  • A text file at /llms.txt, or under a section such as /docs/. Plain markdown, written for language models rather than browsers, so a crawler can read it without working through HTML.
  • One required line: a heading with the site's name. After it come a short summary and lists of links, each with a note.
  • A companion idea: a clean markdown copy of each page at its own address with .md added, so a tool can read the words without the menus and scripts.
  • A map, not a rule. Unlike robots.txt, it grants or blocks nothing.

What the evidence says

SourceWhat it found
Google Search guidanceGoogle says llms.txt does not help or harm visibility or rankings in its search. Other systems may use it.
Ahrefs, June 2026Across 137,210 Ahrefs Web Analytics domains in May 2026, about 28% published the file, and 97% of those files got no requests that month.
The same studyOf the requests that did arrive, coding assistants and OpenAI's crawler sent more than any AI search bot.

This does not establish an AI-search ranking benefit. The Ahrefs sample skews towards technical and SEO-aware sites; its traffic figures do not represent every website or AI tool. Google’s guidance is specific to Google Search.

The map and the pages have different jobs

Our file introduces the site and links to services, industries, public work, articles and team profiles. Each link has a short description. The pages carry the detail behind those descriptions.

A map can make material easier to find without making it more useful. Publishing the file creates another representation of the site to keep accurate. That maintenance belongs in the decision to add it.

What we publish, and why

Our content pages also answer with .md added to their address. The page you are reading answers at /blog/llms-txt.md.

Two panels. Left: our Content and SEO service page in a browser, a dark header with the headline Give your content a clear place to work. Right: the same address with .md added, the same page as markdown: a heading, a one-line summary, then the first section in plain text.
One of our service pages, and the same page with .md added.
  • Made from the same files as the pages. Shared source reduces duplication, but generated copies still need checking against the rendered pages.
  • Notes stay out. The comments we write for each other inside a content file never reach the copy.
  • Private pages stay private. A case study behind our access form answers with a short note on how to request it. The llms.txt file does not list it.
  • Each page points to its copy from its head, as the llms.txt proposal recommends, so a tool that reads one page can find the plain version.

Why we did it: once the content lived in files, both cost little to build. And a buyer's team that pastes our pages into a comparison gets our exact words and links. Ranking was not the reason.

Reading a page and acting on it are different jobs

A plain-text copy gives an assistant the account we publish. A buyer may also ask it to compare suppliers or find the right service. That task depends on the pages and links the index leads to.

Google’s browser-agent guidance describes agents using HTML, screenshots and the accessibility tree. Clear headings, named links and stable controls help those readers understand a page. An llms.txt file cannot supply a missing explanation or make an unclear action understandable.

On this site, public case studies describe the work and service pages define the scope we can discuss. An enquiry still goes through the contact form. The plain-text version carries the published account; it grants no additional access or permission to act.

Should your site have one?

The case is stronger when there is a known reader for the file and the site can maintain it from its existing content. Developer documentation is one such context. A promise of better search rankings is a different claim, and the evidence above does not support it.

Accuracy and access still matter. A generated summary can drift from its page, and a public index can expose an address that should stay private. The extra format carries the same publishing responsibilities as the site.

Our content and SEO page covers the search work we do for marketing teams.

llms.txt is a map for readers who have already decided to visit. The pages still have to be worth reading when they arrive.

Deciding what your site should do for search and AI tools?

Tell us what your site runs on and who needs to find it. We will say what is worth adding, and what would only be busywork.

Talk to us about your site