Automating llms.txt generation for multi-version documentation sites

Developer portals and knowledge bases often contain documentation for multiple product versions, including legacy ones that are no longer actively maintained.

When LLMs try to extract information from these sites, they can struggle to identify which content is current and relevant.

The llms.txt file offers a solution by providing LLMs with clear signposts to the most important content on your site.

However, manually maintaining this file, especially across evolving documentation with multiple versions, quickly becomes an overhead on Technical Authors.

Building a solution

We built an app that automatically generates llms.txt files for documentation sites.

It works in two ways:

• Point it to your documentation navigation structure file, or
• Let it scan your site to discover the content hierarchy

The tool intelligently identifies current documentation and previous versions, creating a clean, LLM-friendly sitemap.

For now, we’re using it internally to streamline our own documentation and on projects. We’re also considering integrating it into our “Managing Docs with AI” and “Writing for Humans and AI” training courses to help others solve the same problem.

Most importantly, it was a fun build; one of those projects that solves a real pain point while teaching us something new along the way.