Does llms.txt replace robots.txt?

No. robots.txt tells crawlers what they may fetch. llms.txt suggests what is worth reading. They do different jobs and you need robots.txt either way. If robots.txt blocks a bot, an llms.txt file will not let it in. Check that every page listed in llms.txt is allowed in robots.txt for the AI crawlers you care about.

Does Google use llms.txt?

No. Google's guide on generative AI search says Google Search ignores llms.txt files, and that creating one will neither harm nor help your visibility or rankings in Google Search. It also says you do not need special machine-readable files or Markdown to appear in Google's AI features. If Google AI Overviews are your main goal, spend the time elsewhere.

Where do I put llms.txt?

At the root of your domain, so it loads at https://yourdomain.com/llms.txt and returns a 200 status. The spec at llmstxt.org also allows a file at a sub-path, covering the pages under that path, which suits a docs folder. Make sure robots.txt does not block it and that your CDN does not challenge bots requesting it.

How do I add an llms.txt file?

Write a Markdown file with your company name as an H1, a one-line summary in a blockquote, then H2 sections that list 10 to 30 key pages as links with a short note after each. Upload it to your site root. Most CMSs allow a static file or a plugin. It takes about 30 minutes, plus a monthly check that the links still work.

Does llms.txt help you get cited by ChatGPT or Claude?

We have seen no public evidence that it does. Ahrefs found that 97% of llms.txt files on 137,210 sites got zero requests in May 2026. The fetches that did happen came mostly from non-AI bots, with GPTBot and Claude Code the top named AI tools. It is cheap to add. Do not expect citations from it.

Library / Visibility / Oct 28, 2026

llms.txt: What It Is, Who Reads It, How to Write One

llms.txt is a Markdown file at the root of your domain that points AI tools to your most useful pages, with a short note on each. It was proposed in September 2024. Writing one takes about 30 minutes. Evidence that AI engines read it is thin, so treat it as cheap hygiene, not a citation lever.

Definition: llms.txt is a proposed standard for a Markdown file, usually served at /llms.txt, that gives large language models a short, curated map of a site’s most important content.

Who proposed llms.txt?

The spec was first published on September 3, 2024 by Jeremy Howard. The idea is simple. A model reading your site has a limited context window and no idea which of your URLs matter. A short file at a known address tells it.

What goes in an llms.txt file?

The spec at llmstxt.org sets the sections in a fixed order:

An H1 with the project or site name. This is the only required section.
A blockquote with a short summary.
Optional paragraphs or lists with more detail, without headings.
Zero or more H2 sections, each a list of links with an optional note after a colon.
An optional section titled “Optional” for links an agent can skip when short on context.

The spec also proposes a clean Markdown copy of each page at the same URL with .md added, for example /pricing.md or /pricing.html.md.

A minimal file for a fictional company:

# Example Analytics

> Product analytics for B2B SaaS teams. Plans from $49 per month.

## Product
- [Pricing](https://example.com/pricing): plans, limits and annual discount
- [Integrations](https://example.com/integrations): 40 native connectors

## Docs
- [Quickstart](https://example.com/docs/quickstart): install in 10 minutes
- [API reference](https://example.com/docs/api): REST endpoints and auth

## Optional
- [Changelog](https://example.com/changelog): release notes since 2023

Does anyone actually read llms.txt?

This is the honest part. Ahrefs studied 137,210 domains using its web analytics in May 2026:

28% published an llms.txt file. Ahrefs calls this an upper bound, because its customers skew technical.
97% of those files received zero requests that month.
Of the requests that did arrive, 96% came from bots, and 19.5% of fetches came from named AI tools. GPTBot was the top AI fetcher. Claude Code was second.

Google is explicit. Its guide says Google Search ignores llms.txt files and that creating one “will neither harm nor help your site’s visibility or rankings in Google Search” (Google Search Central).

The pattern in the data: coding tools and agents fetch llms.txt more than answer engines do. That makes it most useful for developer products, APIs and documentation sites, where an engineer’s coding assistant may read it.

Should you add an llms.txt file?

Your situation	Verdict	Effort
Developer product, API or public docs	Yes. Coding agents are the readers most likely to fetch it	2 hours, with .md page copies
B2B SaaS marketing site	Optional. Add it after crawler access and answer-first pages are fixed	30 min
Ecommerce catalog	Low priority. Product feeds and structured data matter more	30 min
Your goal is Google AI Overviews	Skip for that goal. Google ignores it	n/a

llms.txt vs robots.txt vs sitemap.xml

File	Job	Who reads it	Format
robots.txt	Allows or blocks crawling, per bot	Search and AI crawlers	Plain text directives
sitemap.xml	Lists every URL you want indexed	Search engines	XML
llms.txt	Points to the few pages worth reading first, with context	Some AI tools and agents	Markdown

robots.txt is the one that decides access. A blocked crawler never gets to your llms.txt. See agent readiness for the bot-by-bot rules.

How do you write one in 30 minutes?

Pick 10 to 30 pages. Pricing, product, integrations, docs, and the reference pages you most want quoted. Skip tag archives and anything behind a login.
Write the H1 and the summary. One sentence. What you sell, to whom, and one number.
Group links under H2s. Product, Docs, Company. One line of context after each link.
Upload to the root. Confirm https://yourdomain.com/llms.txt returns 200.
Check monthly. Broken links in the file are worse than no file.

llms.txt does not replace the work that gets you cited. It is one small part of generative engine optimization.

What we check in the Growth Scan

Check	Pass	Warn	Fail
llms.txt present and non-empty (Agent Readiness)	Returns 200 with content	Missing, or returns 200 and is empty	n/a
H1 present	First line is an H1 with the site name	H1 missing	n/a
Links resolve	Every listed link returns 200	1 or more broken links	More than half broken
Listed pages crawlable	No listed page blocked in robots.txt for AI search crawlers	n/a	A listed page is blocked

A missing llms.txt is a Warn, never a Fail. We do not score what we cannot show matters. Fix: add llms.txt (30 min).

Run a free Growth Scan on your site to see where you stand.

Sources

Questions

Does llms.txt replace robots.txt?: No. robots.txt tells crawlers what they may fetch. llms.txt suggests what is worth reading. They do different jobs and you need robots.txt either way. If robots.txt blocks a bot, an llms.txt file will not let it in. Check that every page listed in llms.txt is allowed in robots.txt for the AI crawlers you care about.
Does Google use llms.txt?: No. Google's guide on generative AI search says Google Search ignores llms.txt files, and that creating one will neither harm nor help your visibility or rankings in Google Search. It also says you do not need special machine-readable files or Markdown to appear in Google's AI features. If Google AI Overviews are your main goal, spend the time elsewhere.
Where do I put llms.txt?: At the root of your domain, so it loads at https://yourdomain.com/llms.txt and returns a 200 status. The spec at llmstxt.org also allows a file at a sub-path, covering the pages under that path, which suits a docs folder. Make sure robots.txt does not block it and that your CDN does not challenge bots requesting it.
How do I add an llms.txt file?: Write a Markdown file with your company name as an H1, a one-line summary in a blockquote, then H2 sections that list 10 to 30 key pages as links with a short note after each. Upload it to your site root. Most CMSs allow a static file or a plugin. It takes about 30 minutes, plus a monthly check that the links still work.
Does llms.txt help you get cited by ChatGPT or Claude?: We have seen no public evidence that it does. Ahrefs found that 97% of llms.txt files on 137,210 sites got zero requests in May 2026. The fetches that did happen came mostly from non-AI bots, with GPTBot and Claude Code the top named AI tools. It is cheap to add. Do not expect citations from it.