llms.txt is a proposed web standard — a single Markdown file you place at the root of your domain (yourdomain.com/llms.txt) that gives AI systems a clean, curated map of your most important content. Think of it as a counterpart to robots.txt and sitemap.xml, but written for large language models rather than search crawlers.
It’s an emerging, low-competition opportunity: most sites don’t have one yet, so publishing a good llms.txt is an easy way to make your content easier for AI to find and understand correctly.
What problem does llms.txt solve?
When an AI system tries to understand your site, it faces noisy HTML: navigation, ads, scripts, cookie banners, and boilerplate that obscure the actual content. llms.txt cuts through that by offering a concise, human-readable, Markdown-formatted index of:
- What your site/organization is
- Your most important pages, grouped and described
- Links to clean, content-rich versions of those pages
It doesn’t replace structured data or a sitemap — it complements them by giving AI a curated, prioritized starting point.
What goes in an llms.txt file
The format is intentionally simple Markdown:
# Your Brand
> A one-line description of what your company does.
Optional short paragraph giving context an AI should know about your brand,
category, and key facts.
## Docs
- [Getting started](https://example.com/docs/start): How to set up the product
- [API reference](https://example.com/docs/api): Full REST API documentation
## Guides
- [Pricing](https://example.com/pricing): Plans and pricing details
- [Security](https://example.com/security): Compliance and data handling
## Optional
- [Changelog](https://example.com/changelog): Release history
Key conventions:
- Start with an
# H1of your brand name and a>blockquote one-liner. - Group links under
##section headings. - Each link should have a short description after the colon so the model knows what it is.
- An
## Optionalsection signals lower-priority links that can be skipped when context is limited.
A related convention: llms-full.txt
Some sites also publish llms-full.txt — a single file containing the full text of key pages concatenated together, so an AI can ingest your core content in one fetch. Use llms.txt as the curated index and llms-full.txt as the expanded content dump.
How to create and publish llms.txt
- Pick your most important pages. Documentation, key product/category pages, pricing, and authoritative guides. Prioritize clarity over completeness.
- Write clean descriptions. One concise line per link describing what it is and why it matters.
- Group logically. Use sections (Docs, Guides, Product, Company) that mirror how someone would reason about your site.
- Generate the file. You can hand-write it or use a tool — try the llms.txt generator to scaffold one from your site.
- Publish at the root. Upload it to
yourdomain.com/llms.txtso it’s discoverable. - Keep it current. Update it when your important pages or positioning change — treat it like any other crawlable asset.
Does llms.txt actually work?
It’s an emerging standard, and adoption by AI systems is still evolving — treat it as a low-cost, forward-looking signal rather than a guaranteed ranking lever. The upside is asymmetric: it’s quick to publish, it can only help comprehension, and being early positions you well as adoption grows. Pair it with the fundamentals that AI systems already act on today — authority, clean structure, structured data, and crawlability.
Frequently Asked Questions
What is an llms.txt file?
llms.txt is a proposed standard: a Markdown file at the root of your domain that gives AI systems a curated, prioritized map of your most important content, with short descriptions of each linked page. It helps models find and understand your content without wading through noisy HTML.
How is llms.txt different from robots.txt and sitemap.xml?
robots.txt tells crawlers what they may access, and sitemap.xml lists URLs for search indexing. llms.txt is written for AI models — it’s a human-readable, curated, described index of your key content, optimized for comprehension rather than crawl control.
Where do I put the llms.txt file?
At the root of your domain, accessible at yourdomain.com/llms.txt, just like robots.txt.
What is llms-full.txt?
llms-full.txt is a companion file that concatenates the full text of your key pages into one document, so an AI can ingest your core content in a single fetch. llms.txt is the curated index; llms-full.txt is the expanded content.
Do AI engines actually use llms.txt yet?
Adoption is still emerging, so treat it as a low-cost, forward-looking signal rather than a guaranteed ranking factor. It’s quick to publish, can only improve comprehension, and positions you well as support grows — best paired with the fundamentals AI systems already rely on.