llms.txt is a proposed web standard — a single Markdown file you place at the root of your domain (yourdomain.com/llms.txt) that gives AI systems a clean, curated map of your most important content. Think of it as a counterpart to robots.txt and sitemap.xml, but written for large language models rather than search crawlers.
It’s an emerging, low-competition opportunity: most sites don’t have one yet, so publishing a good llms.txt is an easy way to make your content easier for AI to find and understand correctly.
What problem does llms.txt solve?
When an AI system tries to understand your site, it faces noisy HTML: navigation, ads, scripts, cookie banners, and boilerplate that obscure the actual content. llms.txt cuts through that by offering a concise, human-readable, Markdown-formatted index of:
- What your site/organization is
- Your most important pages, grouped and described
- Links to clean, content-rich versions of those pages
It doesn’t replace structured data or a sitemap — it complements them by giving AI a curated, prioritized starting point.
What goes in an llms.txt file
The format is intentionally simple Markdown:
# Your Brand
> A one-line description of what your company does.
Optional short paragraph giving context an AI should know about your brand,
category, and key facts.
## Docs
- [Getting started](https://example.com/docs/start): How to set up the product
- [API reference](https://example.com/docs/api): Full REST API documentation
## Guides
- [Pricing](https://example.com/pricing): Plans and pricing details
- [Security](https://example.com/security): Compliance and data handling
## Optional
- [Changelog](https://example.com/changelog): Release history
Key conventions:
- Start with an
# H1of your brand name and a>blockquote one-liner. - Group links under
##section headings. - Each link should have a short description after the colon so the model knows what it is.
- An
## Optionalsection signals lower-priority links that can be skipped when context is limited.
A related convention: llms-full.txt
Some sites also publish llms-full.txt — a single file containing the full text of key pages concatenated together, so an AI can ingest your core content in one fetch. Use llms.txt as the curated index and llms-full.txt as the expanded content dump.
How to create and publish llms.txt
- Pick your most important pages. Documentation, key product/category pages, pricing, and authoritative guides. Prioritize clarity over completeness.
- Write clean descriptions. One concise line per link describing what it is and why it matters.
- Group logically. Use sections (Docs, Guides, Product, Company) that mirror how someone would reason about your site.
- Generate the file. You can hand-write it or use a tool — try the llms.txt generator to scaffold one from your site.
- Publish at the root. Upload it to
yourdomain.com/llms.txtso it’s discoverable. - Keep it current. Update it when your important pages or positioning change — treat it like any other crawlable asset.
Where it stands as a standard — stated plainly
Two facts belong side by side here, because most writing on this topic includes one and omits the other.
The proposal is real and specified. llms.txt was proposed by Jeremy Howard in September 2024 and the specification is published at llmstxt.org. It defines the format precisely: an H1 with the project name (the only required section), a blockquote summary, optional prose sections, and zero or more H2-delimited file lists of URLs. It is a genuine spec, not a blog-post convention.
No major search or answer engine has committed to reading it. Google’s own documentation on its AI features states there is no need to create new machine-readable files or AI-specific text files to appear in AI Overviews or AI Mode. That is Google saying, in effect, that this file is not part of its pipeline.
Both statements are true at once, and holding them together is the honest position. llms.txt has real traction in one place — developer documentation, where tools and coding assistants consume it to load a library’s docs into context efficiently. That is a working use case with genuine adoption. Its traction as a brand visibility lever, with the consumer answer engines, remains unproven.
Anyone selling you an llms.txt file as a way to rank in ChatGPT is selling you something no engine has said it uses. Anyone telling you the format is fake is wrong about the spec. Publish it because it is cheap and it may pay off, and because writing one forces a useful exercise — deciding what your twenty most important pages actually are — not because it is a ranking factor.
What to do with the effort instead
If you have an afternoon to spend on making your site legible to AI, the ranking of value looks roughly like this:
- Fix crawl access, agent by agent. Being fetchable by the crawlers that matter is worth more than any file you add, and it is where most sites have an actual defect. See the AI crawlers guide.
- Add the structured data engines demonstrably use. Organization, Product, FAQPage — documented, supported, and consumed today.
- Make your key pages answer their question in the first two sentences. The single highest-leverage change, and the only one on this list that works on every engine.
- Then publish
llms.txt. Twenty minutes, real upside, no downside.
Ordering matters because these compete for the same afternoon. A site with a blocked crawler and a beautiful llms.txt has optimised the step that nobody has confirmed reading while leaving broken the step everybody depends on. If you would rather have the diagnosis than the guesswork, GEO Recommendations produces the prioritised version of this list against your own pages — the specific content edits, schema additions and citation targets, ordered by likely impact.
Does llms.txt actually work?
It’s an emerging standard, and adoption by AI systems is still evolving — treat it as a low-cost, forward-looking signal rather than a guaranteed ranking lever. The upside is asymmetric: it’s quick to publish, it can only help comprehension, and being early positions you well as adoption grows. Pair it with the fundamentals that AI systems already act on today — authority, clean structure, structured data, and crawlability.
Frequently Asked Questions
What is an llms.txt file?
llms.txt is a proposed standard: a Markdown file at the root of your domain that gives AI systems a curated, prioritized map of your most important content, with short descriptions of each linked page. It helps models find and understand your content without wading through noisy HTML.
How is llms.txt different from robots.txt and sitemap.xml?
robots.txt tells crawlers what they may access, and sitemap.xml lists URLs for search indexing. llms.txt is written for AI models — it’s a human-readable, curated, described index of your key content, optimized for comprehension rather than crawl control.
Where do I put the llms.txt file?
At the root of your domain, accessible at yourdomain.com/llms.txt, just like robots.txt.
What is llms-full.txt?
llms-full.txt is a companion file that concatenates the full text of your key pages into one document, so an AI can ingest your core content in a single fetch. llms.txt is the curated index; llms-full.txt is the expanded content.
Do AI engines actually use llms.txt yet?
Adoption is still emerging, so treat it as a low-cost, forward-looking signal rather than a guaranteed ranking factor. It’s quick to publish, can only improve comprehension, and positions you well as support grows — best paired with the fundamentals AI systems already rely on.