llms.txt: a practical guide and its real limits in 2026

What an llms.txt file actually does

Direct answer: an llms.txt is a text summary of your business, published at the root of your site, written for an AI assistant rather than a visitor. It takes about an hour to produce, costs nothing to host, and guarantees no citation in ChatGPT or Perplexity. The rest of this article explains why, and what matters more in its place.

An llms.txt file is a plain Markdown page, published at the root of a website (example.com/llms.txt), that summarises what a company does and points to its most important pages. Unlike a normal web page, it is written for an AI assistant rather than a browser: no layout, no navigation, no scripts, just structured text made of headings and lists of links.

The idea was proposed in 2024 by developer Jeremy Howard. The starting problem is simple: a language model has a limited context window, and a modern website carries too much noise (navigation, banners, scripts) for an assistant to easily pull out what matters in a single pass. llms.txt offers a shortcut: a one-sentence summary, followed by a curated list of the pages that actually count.

Why the format exists

Three answer engines dominate professional queries today: ChatGPT, Perplexity and Google's AI Overviews. Each sends its own crawler (GPTBot, PerplexityBot, Google-Extended) that browses the web in a more targeted, faster way than a traditional search engine. A well kept llms.txt gives them, in theory, a direct map of the pages worth prioritising instead of an entire site to crawl.

That is an attractive promise for a small business that has neither the time nor the budget for a full content strategy: one file, updated in a few minutes, meant to steer bots that otherwise know nothing about the rest of the site. Here is a definition that stands on its own, out of context: an llms.txt is a company summary written for a machine rather than a human, published at a fixed address, that AI assistants can consult before crawling the rest of the site.

The format has never been officially adopted by a major answer engine. OpenAI, Anthropic and Google have published no documentation confirming they rely on it to steer their crawlers. It remains a community convention, pushed more by third-party tools (generators, validators, CMS plugins) than by the companies that actually run the models.

llms.txt, robots.txt and sitemap.xml: three files, three jobs

The three files look alike (root of the site, plain text) but answer different questions. Mixing them up is the most common mistake among teams discovering the topic.

One point is worth clarifying right away, since it comes up often in client conversations: llms.txt blocks nothing. To stop a specific bot from training on your content, robots.txt is the file to edit, with a directive targeting that bot.

Writing an llms.txt in six steps

1. List the pages that actually matter. Between 10 and 30 pages: the homepage, the offer, case studies, cornerstone articles. Not the legal notice, not tag pages.
2. Write a one-sentence summary. This is the line most likely to be quoted if a model cites the file: it has to stand on its own, out of context.
3. Structure it in Markdown. A level-one heading with the company name, a blockquote for the summary, then level-two sections.
4. List links in a standard format. A dash, a Markdown link, a short description, one line per page, to keep it easy for a machine to parse.
5. Publish the file at the root. At the exact address yourdomain.com/llms.txt, as an absolute URL, never in a subfolder.
6. Keep it updated with every publication. A file referencing an article removed six months ago signals neglect, not credibility.

One technical point avoids a common disappointment: the file must stay reachable without authentication and without a redirect. An llms.txt served behind a cookie wall, or redirected to an HTML page, simply does not get read.

Checking whether the file is actually being read

Writing the file does not tell you whether it does anything. The most reliable method is also the simplest: check the server's access logs (or the host's, since both Vercel and Cloudflare expose them) and filter requests to /llms.txt by User-Agent header. Bots that identify themselves honestly do so clearly: GPTBot, ClaudeBot, PerplexityBot, Google-Extended. A request with no recognisable header, or coming from a residential IP address, is almost always a human audit or a third-party tool, not an AI assistant preparing an answer.

For most small business sites, this check takes one command line and a few minutes a month. It is a minimal habit, but far more honest than simply assuming the file is working.

A real example: the llms.txt of tesseractstudio.ch

Rather than staying theoretical, here is the practice. This site's llms.txt fits on one page: a one-sentence summary, three sections (what the studio does, why us, offers and pricing), the list of past projects with precise figures, then the list of blog posts, regenerated automatically with every publication. Nothing more. No decorative sections, no extended sales pitch: language models do not reward length, they look for factual density.

The blog section is generated by the build script at every new article, using a start and end marker in the source file. That is a technical detail, but it illustrates the core principle: an llms.txt that requires a manual update on every publication eventually stops being updated at all.

Mistakes that make an llms.txt useless

The most common one is copying the homepage's marketing copy, superlatives included. A language model handles superlatives ("leading", "best", "innovative") poorly: they carry no verifiable information. The second mistake is length: a file running to thousands of lines stops being a summary and becomes a site to crawl again, exactly what the format was meant to avoid. The third is abandonment: an llms.txt published once and never revisited loses its value within months, as the pages it lists change or disappear.

The fourth mistake, more subtle, is treating llms.txt as an access control mechanism. That is not its role: a bot that deliberately ignores the rules (which happens) is stopped by neither llms.txt nor robots.txt. Both files rely on the bot's voluntary cooperation, not on a technical block. For a restriction that actually holds, a server-side rule (application firewall, IP allow list) is the right tool, not a text file.

When not to bother with an llms.txt

This is the part most guides on the topic skip, and it deserves numbers rather than intuition. In May 2026, Ahrefs analysed the logs of 137,210 domains tracked by its analytics tool: 97 percent of published llms.txt files received not a single request that month. Of the roughly 38,000 domains with a valid file, only about 1,100 saw any visitor at all, human or bot. And among the few requests that did land, 77 percent came not from an AI tool but from SEO audit crawlers or generic bots: Slackbot alone requested llms.txt files more often than PerplexityBot did.

An independent study by Otterly.ai, run over 90 days across more than 62,100 AI bot visits on a test site, reaches the same conclusion: the llms.txt file received only 84 requests, three times fewer than an average content page, and no correlation was measured between its presence and an increase in citations. Google has publicly stated that it does not rely on the format.

The honest conclusion: if producing the file takes an hour and it updates itself, the opportunity cost is zero, and there is no reason not to publish it in case adoption grows. But commissioning a paid audit or a dedicated project purely for an llms.txt is not, today, a defensible investment. That is not where visibility in answer engines is decided.

The calculation changes with team size. A startup with a developer able to generate the file from existing content in an hour has no reason to skip it. A small business that would need to pay an agency by the day for the same task is better off spending that budget on one of the six levers that are actually correlated with measurable citations: page structure, structured data, or content that answers a question directly without a marketing detour. llms.txt is a checkbox, not a strategy.

What actually moves the needle for citations

llms.txt remains one building block among many, and a minor one. The levers that make a measurable difference to citability (page structure, factual density, structured data, server side rendering) are covered in our overview of the six levers of visibility in AI answer engines. For a Swiss SME that wants to know where it stands before spending time on this, the GEO Sprint starts from a concrete audit rather than an isolated text file. Every article the studio publishes on the topic is collected on the blog.