llms.txt Explained: Format, Examples, and Value

llms.txt is a plain-Markdown file at your site root that gives language models a curated, human-readable map of your most useful content. It is a proposal, not a ratified standard - but it is cheap to publish and it forces a useful exercise: deciding which 30 pages actually represent your site.

Last updated: · By SEO Smart Engine Team

What it is

A Markdown file at /llms.txt containing an H1 with your site name, a blockquote summary, and grouped link lists with one-line descriptions. An optional /llms-full.txt carries expanded context for models that can ingest more.

How it differs from robots.txt and sitemap.xml

robots.txt controls access, sitemap.xml lists every indexable URL for crawlers, llms.txt curates and describes a subset for language models. They complement each other; none replaces another.

A minimal working example

Start with '# Site Name', then a blockquote of one or two sentences describing what the site does, then sections like '## Free Tools' and '## Guides' with Markdown links and a short description after each colon. Keep it under a few hundred lines and keep every link live.

What it realistically achieves

No major engine has committed to using it as a ranking or retrieval input. Its practical value today is discoverability of your best pages for agents that do fetch it, plus internal clarity about your content hierarchy. Treat it as low-cost, low-risk hygiene rather than a growth lever.

Maintenance rules

Regenerate it whenever you publish or retire significant pages, never list private or gated routes, and validate that every URL returns 200. A stale llms.txt full of 404s is worse than none.

In-depth guide

A longer, practitioner-level breakdown of llms.txt - written for readers who want the full picture, not just the summary above.

An honest assessment of the format

It is worth being precise about what is known. llms.txt is a community proposal. No major search or assistant provider has publicly committed to reading it as a retrieval or ranking input. Publishing it will not, on its own, produce citations, and any vendor claiming otherwise is overselling.

What it does provide is a curated, machine-readable statement of what your site is and which pages matter, at effectively zero cost, in a format that agents and scrapers can consume without parsing your navigation. As agentic browsing becomes more common, a clean manifest is a reasonable hedge. It also has an underrated internal benefit: the exercise of choosing thirty representative pages exposes content sprawl that no analytics dashboard surfaces.

Treat it as hygiene alongside robots.txt and sitemap.xml, keep it accurate, and set expectations accordingly with stakeholders.

Keeping it correct over time

The failure mode is staleness. A manifest listing retired URLs, renamed tools, and outdated pricing actively misinforms any agent that reads it, which is worse than publishing nothing. Regenerate it as part of the same routine that regenerates your sitemap, and validate that every listed URL returns a 200 rather than a redirect chain or a soft 404.

Never list authenticated or gated routes. A manifest is a public document, and enumerating private dashboard paths hands an attacker a map while offering no discovery benefit, since those pages should be disallowed anyway.

Keep the descriptions honest and specific. One clause explaining what the page does is more useful to a retrieval system than a marketing sentence, and it is far easier to keep accurate.

Free tools to apply this

FAQ

Is llms.txt an official standard?

No. It is a community proposal that a growing number of sites publish voluntarily.

Does llms.txt improve AI citations?

There is no confirmed direct effect. It can aid discovery for agents that read it, which is why many sites publish it anyway.

Should llms.txt list every page?

No. Curate your best public pages. Comprehensiveness is sitemap.xml's job.

Where does the file go?

At the site root, served as text/plain or text/markdown, at https://yourdomain.com/llms.txt.

Related guides

Continue building topical authority with the guides closest to this one.

Recommended for your site

Ranked by topical relevance to this page.

Go deeper

Comparisons, playbooks and use-case breakdowns that build on this topic.