Key points
- llms.txt is a proposed Markdown file at the root of a website that gives AI models a short summary of the site and links to its most useful pages. It was proposed by Jeremy Howard in September 2024.
- It is a proposal, not a standard. No major AI company has said it uses llms.txt to decide what to cite, and Google’s John Mueller compared it to the old keywords meta tag in April 2025.
- It is cheap to publish and does no harm, so it is reasonable to have one, but it is not a substitute for crawl access, clear content and accurate structured data.
Few topics in AI visibility attract as much attention, or as much overstatement, as llms.txt. This article explains what the file is and what is actually known about its use. It is part of our AI search visibility guide.
What llms.txt is
llms.txt is a plain Markdown file placed at a site’s root, for example example.com/llms.txt. It was proposed by Jeremy Howard on 3 September 2024 as a way to give AI models concise, LLM-friendly information about a site and links to more detailed content (llmstxt.org). The idea is that a language model with limited reading capacity can understand a site quickly from one curated file instead of crawling cluttered web pages.
The format
The proposal sets out a simple structure, in this order (llmstxt.org):
- An H1 heading with the name of the site or project. This is the only required part.
- A blockquote with a short summary.
- Optional paragraphs or lists with more detail.
- H2 sections containing lists of links, each written as
[name](url)with optional notes. - An optional section titled “Optional” for secondary links that an AI can skip.
The proposal also suggests offering clean Markdown versions of important pages at the same address with .md added.
A short example
# Newsinterpretation
> English- and Marathi-language news and analysis on cyber security, AI and geopolitics.
## Research series
- [Cyberattacks That Endanger Human Lives](https://newsinterpretation.com/cyberattack-catastrophe-endanger-human-life/): research series
- [AI Search Visibility](https://newsinterpretation.com/ai-search-visibility-guide/): guide to being cited by AI assistants
## Optional
- [About us](https://newsinterpretation.com/about-news-interpretation-initiative/)Does anyone use it?
This is where claims outrun evidence. In April 2025, asked whether AI bots read llms.txt files, Google’s John Mueller said none of the major AI services had said they use it, that server logs showed they did not request it, and that “it’s comparable to the keywords meta tag” (Search Engine Journal). The keywords meta tag was a once-popular way for sites to describe themselves that search engines stopped trusting because it was easy to game.
The format has been adopted mostly by software documentation sites, where developers point AI coding tools at it. For general publishers, there is no public evidence that it changes whether ChatGPT, Claude, Gemini or Perplexity cite a page.
Should you publish one?
Probably yes, but with modest expectations. It takes minutes, carries no known risk, and may become more useful if AI companies adopt it. What it cannot do is make up for blocked crawlers, thin content or inconsistent facts. Spend more time on those; see our guide to AI crawlers and our guide to structured data.
Two practical cautions: keep the file accurate and short, because a long list of every page defeats its purpose, and make sure your server returns it as plain text rather than redirecting it or blocking it behind a firewall.
A worked example
Newsinterpretation publishes an llms.txt file generated by Rank Math and edited to point to our research series. NI Geovanta’s AI Readiness Report checks whether the file exists and can be read, along with robots.txt, sitemaps and structured data. We are watching NI Geovanta’s AI visit log and will update this article if AI crawlers start requesting the file.
Frequently asked questions
What is llms.txt?
llms.txt is a proposed Markdown file at the root of a website that summarises the site for AI models and links to its most important pages. Jeremy Howard proposed it in September 2024.
Does ChatGPT or Google use llms.txt?
No major AI company has said it uses llms.txt to choose sources. Google’s John Mueller said in April 2025 that AI services had not said they use it and compared it to the keywords meta tag.
Is llms.txt the same as robots.txt?
No. robots.txt tells crawlers what they may access. llms.txt is a curated guide to a site’s content and has no power to allow or block anything.
Should my website have an llms.txt file?
It is reasonable to publish a short, accurate one because it costs little, but it should not be relied on to improve AI visibility.
Last reviewed: 22 October 2026. Sources are linked inline.
Disclosure: NI Geovanta is developed by Newsinterpretation, the publisher of this article. It is used here as a worked example; other tools can perform similar checks.
