Every few months a new file gets proposed for the root of your website. First it was robots.txt, then sitemap.xml, then security.txt. The latest one clients keep asking us about is llms.txt.
Last month a client who runs a chain of physiotherapy clinics in Pune forwarded us a LinkedIn post with the subject line "do we need this??". Two question marks. The post claimed that websites without an llms.txt file would "disappear from AI search". We get some version of that message every week now, so here's the answer we gave her, slightly tidied up.
What llms.txt is
It's a proposal, first published in 2024, for a plain Markdown file at yoursite.com/llms.txt. The idea is simple. Large language models struggle with normal web pages because of menus, cookie banners, scripts and layout code. So you give them a short, clean summary instead: who you are, what you offer, and links to the most useful pages, ideally in plain text versions.
A typical file has a title, a one-paragraph summary, and a few sections of links with a short note next to each.
What it does today
This is the honest part. llms.txt is a community proposal, not a standard, and none of the big AI companies has publicly committed to reading it when answering questions. Some developer tools and documentation platforms use it, and some AI coding assistants will read it if you point them at it. But there is no evidence that adding one makes a business more likely to be recommended by an AI assistant.
So if someone is selling you an "llms.txt optimisation package", be careful.
Why we add one anyway (sometimes)
For most small business websites, we treat it as optional and cheap. Where it does make sense:
- Software products and APIs. If developers use AI tools to work with your product, a clean llms.txt pointing to your docs genuinely helps those tools.
- Sites with lots of documentation. Help centres, knowledge bases and product manuals benefit from a tidy index.
- Clients who want to be early. It takes about twenty minutes to write and costs nothing to host.
Where we don't bother: a five-page brochure site for a local clinic. For the Pune client, we spent the hour we'd have spent on llms.txt fixing her opening hours, which were wrong on two of her location pages. That will do far more for her, with people and with AI tools alike.
The basics matter far more
Whether an AI system reads llms.txt or not, it almost certainly reads your actual pages. The things that help there are the same things we've written about before:
- Clear page titles and headings that say what the page is about.
- Key facts in text: prices, service areas, opening hours, contact details.
- Structured data for your business, products and FAQs.
- Fast pages that don't hide content behind heavy JavaScript.
- A sensible
robots.txtthat doesn't accidentally block crawlers you want.
If all five are in good shape, an llms.txt file is a nice extra. If they're not, it won't make up for it.
If you want to try it
Keep it short and human-readable. Write a two or three sentence summary of your business, then list your most important pages with a one-line description each. Update it when you add or remove services. And treat it like any other page: if it's out of date, it's doing more harm than good.
We did add one to our own site, mostly out of curiosity. So far, the only visitors who have requested it are a couple of developer tools and, we suspect, other agencies checking whether we have one. We'll keep watching whether the major AI providers adopt it. If they do, we'll be the first to tell our clients. Until then, think of it as a low-cost experiment, not a strategy.
