Every few months a new file appears that you are told your website must have. The current one is llms.txt. The pitch is that AI assistants will read it and understand your business properly.
I have now added this file to several sites, including my own, and built a tool that generates it. So this is the honest version, including the part most articles skip.
What the file actually is
Jeremy Howard proposed /llms.txt on 3 September 2024. It is a markdown file at the root of your domain, in the same position as robots.txt, but it is written for language models rather than crawlers deciding what they may fetch.
The structure is simple. You need an H1 with the site name, and that is the only section the specification actually requires. Everything else is optional: a blockquote holding a short summary, some prose, and H2 sections containing lists of markdown links with optional notes after a colon.
That is the whole format. There is no schema to validate against and no registry to submit to. You write a file, you upload it, and you are done in about ten minutes.
Who is actually publishing one
Rather than repeat what other posts assert, I checked. In August 2026 I requested /llms.txt from sixteen well-known domains and recorded the response.
Eleven returned a real file: stripe.com, docs.stripe.com, cloudflare.com, developers.cloudflare.com, vercel.com, nextjs.org, svelte.dev, zapier.com, hubspot.com, shopify.com, and docs.anthropic.com.
Three returned 404: anthropic.com, platform.openai.com, and astro.build. Two, openai.com and perplexity.ai, returned 403 to my request, which tells me their edge blocked me rather than telling me anything about the file.
Two things stand out. The first is that adoption is real and skews heavily towards developer documentation, which is exactly the content people paste into an assistant. The second is that Anthropic publishes one on its docs subdomain but not on its main marketing domain, which is a reasonable read of where the format is useful.
The part most articles skip
Publishing a file is not the same as anything consuming it.
Read the specification page carefully and you will notice it makes no claim about which AI systems support the format. It describes what the file should contain and why that would help a model working inside a limited context window. It does not say that ChatGPT, Claude, Gemini or Perplexity fetch it, because as far as any public documentation goes, none of them have committed to doing so.
So when someone tells you llms.txt improves your AI visibility, ask them how they know. The honest answer today is that we have a well-designed convention with real adoption and no published evidence from the major AI vendors that it changes what an assistant says about you.
How to find out for yourself
You do not have to take my word for it or anyone else’s, because your server already knows. Look in your access logs for requests to /llms.txt and note which user agents made them.
If you are on Cloudflare, the same question is answerable from the dashboard analytics for that path. If nothing has ever requested the file, that is your answer for your site, and it is worth more than any general claim including this article.
This is the habit worth building. Regulation and hype both move faster than evidence, and your own logs are the only source that is specific to you.
So should you write one?
Yes, with clear eyes about why.
It costs one file and ten minutes. If the format gets picked up, you are already there. If it does not, you have lost almost nothing, and you have been forced to write a plain description of what your business does and which pages matter most, which is a useful exercise on its own.
What you should not do is treat it as a substitute for the things that demonstrably work. If you want the file without doing it by hand, our free llms.txt generator builds one from your site’s own metadata, and the Agent-Readiness Score checks whether the rest of your site is legible to machines.
What matters more
If your goal is for AI assistants to describe your business correctly, three things carry more weight than this file today, and all of them are verifiable.
Check that you are not blocking the crawlers. Plenty of firms have Disallow rules that silently exclude GPTBot or ClaudeBot, and no file at your root will fix that.
Check that your important content is in the server-rendered HTML. If your services only appear after JavaScript runs, most crawlers never see them, and the pages that describe what you sell are invisible.
Then check that your pages answer questions directly. Structured data helps a machine identify what your organisation is, but a page that states a price, a scope and a timeline in plain sentences is what gets quoted back to someone asking about your category.
Write the llms.txt file. Just put it in the right order of priority, and be suspicious of anyone who tells you it is the reason you are not showing up.