I’ll admit, I’m probably slightly sadder than most people when it comes to thinking about AI, Google and the technicalities of how websites are being found, but this was something that came into my head recently: do websites actually need an llms.txt file?
It’s something that has started appearing more and more in conversations around AI optimisation, and if you use Yoast on WordPress you may already have noticed that there’s now an option to create one. So I thought it was worth looking at what it actually does, whether Google uses it and whether you should bother switching it on.
First of all, what is llms.txt?
An llms.txt file is a relatively new proposal designed to give large language models, or LLMs, a simpler way of understanding the important information on your website. You’ll normally find it at yourwebsite.co.uk/llms.txt.
The idea is fairly straightforward. Websites are built for people, so they contain menus, layouts, scripts, images, navigation and all sorts of other elements that are useful to somebody browsing a website but aren’t necessarily the cleanest way for an AI system to understand what the site contains. An llms.txt file can provide a much simpler version of that information, pointing AI systems towards useful pages and content.
Think of it more as a signpost for AI than some magical new SEO file. The original llms.txt proposal was launched in 2024 and describes it as an LLM-friendly Markdown document containing background information about a website and links to useful content. Importantly, it remains a proposed standard, so there isn’t a rule saying that AI platforms have to use it.
What does it actually do?
In simple terms, it can give an AI tool a clearer indication of what your website is about and which pages might be particularly useful. That might include your main services, key product pages, useful guides, important information about your business and other cornerstone content.
In theory, that makes it easier for an AI system that supports llms.txt to find the information that matters. What it doesn’t do is guarantee that ChatGPT, Gemini, Claude or another AI platform is going to recommend you, and simply having an llms.txt file doesn’t suddenly mean your website has been “optimised for AI”. There is still quite a lot more to it than that.
What does Google say about llms.txt?
This was the bit I was particularly interested in. Google clarified its position in June 2026 and it’s actually pretty definitive.
Google says:
“You don’t need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search.”
Google has also confirmed that it doesn’t currently use llms.txt for Google Search, including its generative AI search features, and has said that having an llms.txt file will neither help nor harm your Google rankings.
So if somebody tells you that you need an llms.txt file to appear in AI Overviews or improve your rankings on Google, that isn’t what Google itself is saying. Google’s advice continues to be much more familiar: create useful, original content, make your website technically accessible and make sure people can easily understand what you do.
Which, perhaps slightly disappointingly for those looking for a magic AI optimisation button, sounds quite a lot like good SEO has always sounded.
Can you add llms.txt through Yoast?
Yes, and this is probably why I think there’s very little downside to having one. Yoast now includes an llms.txt feature for free, and if you’re using an up-to-date version you can go to Yoast SEO → Settings → Site Features → AI tools → llms.txt and switch it on.
Yoast creates the file for you and automatically updates it, so you’re not committing yourself to another technical job that needs maintaining every month. That changes the question slightly for me. If Google doesn’t require it, but Yoast lets you add it in seconds and maintain it automatically, why wouldn’t you switch it on?
So, do I think you need one?
Strictly speaking, no. Google doesn’t need it, Google isn’t currently using it and adding one isn’t going to improve your Google rankings overnight.
But would I add one? Yes.
We’ve enabled it on our own website. It takes seconds, it costs nothing if you’re already using Yoast and it gives AI platforms another clean way of understanding the website if adoption of llms.txt grows.
I’d just keep it firmly in perspective. I wouldn’t recommend somebody spend hundreds of pounds having an llms.txt file developed or treat it as the centrepiece of an AI optimisation strategy, because there are far more important things to get right first.
What actually matters for AI optimisation?
This is the interesting bit for us at the moment. We’ve been doing a lot of work recently on our own website around the way our services, sectors, experience and expertise are structured and explained, and we’re already seeing some of that information appearing within AI-generated search results within days of putting it live.
That doesn’t mean llms.txt is responsible for it. What it does suggest is that AI optimisation is much broader than simply adding one file to your website.
You need clear information about what you do, useful content that properly answers the questions people are searching for, a logical website structure and enough context for search engines and AI systems to understand the relationship between your business, your services, your expertise and the subjects you talk about. And, ultimately, you still need the website to be genuinely useful to the person who lands on it.
That last bit hasn’t changed.
What about using AI to create the content itself?
That was another question I had. If we’re talking about optimising websites for AI, does Google care if AI has actually helped create some of the content?
Again, Google’s position is relatively sensible. Using AI isn’t against its guidelines. Google says that the appropriate use of AI or automation is fine, and what it cares about is the quality and purpose of the finished content, rather than whether every word was physically typed by a person.
Where you run into problems is using AI to churn out hundreds of thin, repetitive pages purely to manipulate search rankings. That’s very different from using AI to research a subject, help structure an article and then applying your own experience, knowledge and opinion to it.
Which is, incidentally, exactly how this article came about. I wondered about llms.txt, I checked what Google actually says about it, and this is my conclusion:
Switch it on if you’ve got the option, but don’t mistake it for an AI optimisation strategy.
Need help optimising your website?
AI is changing the way people find businesses online, but many of the fundamentals remain the same. Your website needs to clearly demonstrate what you do, answer the questions your customers have and give search engines and AI systems information they can confidently understand.
If you’d like us to take a look at how well your website is doing that, get in touch with us.