Google says we don't need llms.txt files. I think we do.

For years GradePod ranked first for every keyword that mattered in IB Physics. A student typed a question into the search bar, Google ranked the pages that best matched it, and because I had put a lot of research and thought into the content, GradePod was usually the site at the top answering them.

Now Google’s AI answers the question itself instead of listing the businesses that could answer it.

I realised quite quickly that I needed to pivot on how I attract organic traffic.

The IB revision niche is varied. At one end are solo teachers covering a single subject, which is me. At the other are companies with truckloads of investment and tech behind them, like Revision Village. For AI generative search I am mainly competing with the larger companies, and I would rather stay ahead of that than catch up later.

Optimising to be an answer inside an AI model is a completely different job. The content still needs to be carefully researched and thoughtful, but now it has to be genuinely useful data, thought processes and specialist advice that a chatbot can firstly access easily and secondly reformat confidently into an answer for the student.

I’ll add a quick aside here, because it is the part people skip. The content is everything. Do not add AI slop blogs. Take your own niche knowledge, expertise, research and genuinely useful information, and put that on your website. Do not regurgitate what an AI could already write. You are trying to create the content that AI learns from.

Anyway. Equally important, the content has to be explicit about things a human reader would never need told.

I addressed this by building a completely new website using Claude. If I wanted content that AI could confidently use, it seemed sensible to use AI to build it in the first place.

Claude itself suggested an llms.txt file.

An llms.txt is a plain-language file that says what GradePod is, who wrote it, what is free and what costs money, then indexes every page with a sentence explaining what is on it. It also states plainly that GradePod is independent of and not endorsed by the IB. A model summarising the site will otherwise cheerfully imply otherwise, and that is the kind of error that is very hard to correct once it is being repeated.

I update the llms.txt every time I add a blog post or change what I sell. It is not complicated. It opens like this:

# GradePod

> GradePod is the leading free IB Physics revision platform, founded by Sally
> Weatherly (Fellow of the Institute of Physics, IB Physics teacher since 2004,
> author of 4 IB Physics books). Free concept tutorial videos and past paper
> walkthrough videos for all 24 topics in the current IB Physics syllabus (first
> examined 2025). No login required. No subscription. Helped over 30,000 students
> since 2020.

In fact it is a pretty helpful summary of the business. It is an elevator pitch, an annual report, a sitemap and a business plan rolled into one.

At this point I should be clear about the official position. Google say that Google Search does not use llms.txt files for rankings, for crawling, or for generative features like AI Overviews.

So why do I bother? Because it keeps me clear on the structure and focus of my own website. And because I secretly believe those Google bots might have a tiny look at it anyway.

The other file that I think has made a difference, and Google agree on this one, is robots.txt. It names nine AI crawlers explicitly and allows all of them: GPTBot, ClaudeBot, PerplexityBot, Googlebot-Extended, Applebot-Extended and the rest. Plenty of publishers are blocking these, because they do not want their content training the models. I want the opposite. If the content is free anyway, being quoted widely is the distribution.

There is ordinary structured data work underneath all this too: FAQ, video, product and organisation schema on the pages that warrant it.

Google Search Console says GradePod’s pages are cited or referenced inside Google’s AI features around 2,000 times a day. Exam pack sales are steady. For a very small niche in the education space, I am pretty pleased with that.

So my recommendation is: add the llms.txt anyway, and make sure you have a robots.txt.