How to help AI understand the important parts of your website

Your website's content is written for your users: it can be personable, creative and deeply engaging, helping spark curiosity for the human mind.

But to AI agents, it can become unnecessary noise.

AI reads and understands your site in a different way: it doesn’t need that curiously colourful content or flourish of words: it needs facts, structured content and clear direction.

Any web developer can create a llms.txt file for a website - but what about when your website grows and it needs changing?

What is a llms.txt file?

A llms.txt file is a simple text file that sits at the root of your site, and is formatted with markdown.

While not an official standard yet, it is an emerging convention.

Its purpose is to act as a signpost or roadmap that helps AI models find your most important and accurate information in a structured and concise manner.

It is important to note too that not every AI model explicitly uses the llms.txt - but it can act as a hint or optimisation to help.

Why do I need a llms.txt file?

You’ve probably heard of SEO - Search Engine Optimisation - but have you heard of GEO? That’s Generative Engine Optimisation - another emerging term.

You spend so much time making your content work for your site for search engines, but GEO is the practice of optimising your content to be cited or recommended within AI-generated answers - these could come from an AI agent like ChatGPT or Claude, or even through Google’s own generated results too.

Your llms.txt file’s purpose is to help make your site’s information easily understood by Large Language Models (LLMs). This can help your site build authority and credibility within the space, and can expand beyond just your actual site but also to related sites, reviews and platforms including YouTube or social media.

With the increase in adoption of AI-based queries across personal and business use, your site needs a llms.txt file to try to help improve your visibility and discoverability.

What should my llms.txt file include?

Your llms.txt file should be concise and direct, using plain and unambiguous language so that AI can more easily process and understand intent.

While your page content may include story telling and personal voice - which is so valuable for your brand identity and human visitor engagement and conversion - this just adds a noisy ambiguity for AI.

Being clear, concise and direct will help your AI better understand your intent.

Your llms.txt should include:

• a single heading 1 (<h1>) to start the file off

• a brief introduction

• sections for key components of your business, with succinct and authoritative links to suitable content

• clear contact information

• any special instructions for AI agents to assist with understanding

If your site has multiple languages, your llms.txt should be written in English, but include references to the site’s available languages, and if suitable, links with clear sections as to what is in each language.

Most importantly, don’t make it a dump of every page on your site like a sitemap - this lacks structure, intent and clarity, and just turns up the noise.

Your llms.txt file should be human-curated.

Why should this be human-authored?

You can use your favourite AI tool to help you write your llms.txt file - but you, as the intentful human, should be the one who publishes the final content.

Automating this with large sites can create a polluted and messy llms.txt file that adds unnecessary weight and confusion for AI to try to understand.

Creating clear and direct structure can help AI reduce the chance of hallucinations when generating content about your site. And the best way to achieve this is by being the curator of your own structure.

There’s no hard and fast one-size-fits-all structure that works for every business and every site: but take a moment to think about what is important for you, your business, your site’s structure and your offering, and use this to guide your human-authored content to better help AI.

Creating your llms.txt in Statamic

We love developing websites with Statamic, and firmly believe in the benefits of a human-authored llms.txt.

But not everyone is comfortable writing markup, and with a content management system like Statamic, content authors need the ability to create and update their llms.txt file without technical barriers.

So we created the llms.txt for Statamic addon - a free and open source addon that adds llms.txt support for Statamic sites.

After installation, you can author your llms.txt file directly in Statamic’s CP, and also control access through Statamic’s flexible permissions and roles.

The editing experience is a familiar WYSIWYG editor that is incredibly friendly for any content author. You can even use Statamic’s link browser to link to other pages of your site too.

We’ve included some validation rules too to help you ensure your authored content does not do things that your llms.txt file should not expect - like nested lists or multiple headings.

Want to get started? Read the full docs, and contact us if you have any queries.

AI is constantly evolving, and the contents and structure of your llms.txt file too should keep evolving. As the recommended standards change, don’t forget to put in the time and effort to keep your llms.txt file up to date with just as much focus as your on-page SEO too.