llms.txt

llms.txt is a proposed standard — a plain-text file in Markdown format placed at a domain's root (at /llms.txt) whose job is to help large language models find and understand a website's most important content. It emerged in response to the growing role of language models and AI assistants in how people access information.

How llms.txt works

The problem it addresses is practical: web pages contain plenty of material irrelevant to a model — menus, scripts, ads, footers — while the context window of AI models is limited. An llms.txt file provides a tidy, stripped-down map of the content: a short description of the site plus a list of links to key pages with notes on what each resource contains. Markdown is easy for both humans and machines to parse. A variant, llms-full.txt, is sometimes used to carry the full documentation content in a single file.

Practical application

llms.txt is especially useful for documentation sites, knowledge bases and large product sites where you want AI assistants to cite and summarize your offering accurately. It is worth distinguishing from robots.txt: robots.txt states what bots must not crawl, whereas llms.txt does the opposite — it actively highlights which content matters most and presents it in an accessible form.

Keep in mind that this is a community-proposed standard rather than an official provider requirement, so adherence depends on the specific tool. Just as an XML sitemap does not guarantee indexing, llms.txt does not guarantee a model will use it — but it is an inexpensive step toward visibility in AI-powered search.