In this page
What makes content fit for AI retrieval, and what disqualifies it
What the audit covers
Audit scope options
Who this is for?
Find out what your AI is working with
Cherryleaf’s RAG readiness audit
You’re deploying an AI assistant. Or you already have one, and the answers it gives aren’t quite right.
Or you’re planning to, and someone has just asked whether your documentation is good enough to train it on.
That question matters more than most teams realise.
Retrieval-Augmented Generation (RAG) is the technique that powers most AI support tools, internal assistants, and intelligent search systems. The AI retrieves content from your knowledge base, help centre, or documentation, then generates an answer based on what it finds.
This means the accuracy of every answer your AI gives is determined by the quality of the content it retrieves from.
The problem
If your documentation is outdated, your AI will give outdated answers.
If your content contradicts itself, your AI will give inconsistent answers.
If your articles are too long, too vague, or poorly structured, your AI may silently skip the most important part, and still give an answer, confidently.
AI tools do not compensate for poor source material. They reflect it, at scale, to every user who asks a question.
What makes content fit for AI retrieval, and what disqualifies it
Most documentation was written to be read by a human in sequence, on a screen, with the ability to scroll back, re-read, and follow links.
AI agents work differently:
- They retrieve in chunks.
- They have hard limits on how much they can read in a single call.
- They don’t always know when they’ve been cut off.
- They produce an answer regardless, whether they had enough information to do so reliably or not.
The common content problems that break AI retrieval
Outdated or inaccurate content
The AI cites the information it finds. If the answer is wrong, the AI answer is wrong.
Articles that are too long
An agent retrieving a 4,000-word article might read only the first 1,500 words and answer from an incomplete picture.
Contradictory content across articles
When multiple articles say different things about the same topic, the AI may blend them, producing an answer that matches none of them exactly.
Missing metadata and poor structure
Without clear headings, topic boundaries, and consistent terminology, retrieval algorithms struggle to identify which content is relevant to which question.
No distinction between audience types
End-user content, administrator content, and developer content mixed together creates retrieval noise: the AI retrieves the wrong answer for the wrong user.
FAQs with vague answers
Short, clear question-and-answer pairs are one of the most reliably retrieved formats. Vague answers to specific questions undermine this advantage.
Content gaps
Questions your users actually ask that aren’t answered anywhere in your documentation. The AI has nothing to retrieve, and so it fabricates.
What the audit covers
Cherryleaf’s RAG Readiness Audit is a structured assessment of your existing documentation. For example: knowledge base, help centre, internal wiki, policy library, or any content source you intend to use with an AI tool.
We assess your content against the criteria that determine AI reliability: accuracy, structure, length, consistency, coverage, and retrievability.
The audit process
Step 1: Scoping call (60 minutes)
We talk with you to understand your AI tool, your content sources, the questions your users typically ask, and your current support or internal knowledge pain points. We agree which content is in scope.
Step 2: Content sample review
We review a structured sample of your documentation. This is typically 50 to 150 articles, pages, or documents, depending on the scope tier you choose. We assess each against a standardised RAG readiness framework covering accuracy, structure, length, consistency, coverage, and metadata.
Step 3: Gap and pattern analysis
We identify the patterns behind the problems: the systematic issues that, once fixed, improve readiness across your entire content set rather than article by article.
Step 4: Prioritised remediation report
You receive a written report with:
- An overall RAG readiness score for your content
- A breakdown of the most common failure patterns
- A prioritised list of remediation actions, from quick wins to structural changes
- Recommendations on content structure, chunking approach, and metadata
- A suggested roadmap for implementation, whether you carry out the work internally or with Cherryleaf
Step 5: Findings walkthrough (60 minutes)
We present the report to your team, answer questions, and help you prioritise next steps.
What you get at the end
- A clear, honest picture of whether your current content is fit for AI retrieval
- A prioritised list of what to fix first, specific to your content
- A remediation roadmap you can act on, with or without Cherryleaf’s continued involvement
- A reusable content quality framework your team can apply to new content going forward
Audit scope options
The right scope depends on how much content you have and how deep you need the analysis to go.
Focused audit
For teams with a single knowledge base or limited documentation set (up to 50 articles or documents)
Covers one content source.
Ideal for teams with a specific AI tool deployment in mind and a relatively contained documentation scope.
Delivers a prioritised report and a 60-minute findings walkthrough.
Standard audit
For teams with a broader documentation set (up to 150 articles or documents, or multiple content sources).
Covers one to three content sources. For example: Customer knowledge base, internal procedures, and product documentation.
Delivers full gap analysis, pattern identification, and a detailed remediation roadmap.
Comprehensive audit
For larger organisations or teams with complex content ecosystems
Covers a larger or more complex content set. Includes additional stakeholder interviews, integration with your existing AI tool or platform provider’s requirements, and an extended implementation planning session.
Not sure which tier fits?
The scoping call will tell us both.
Who this is for
This audit is right for you if:
- You’re planning to deploy an AI support tool, internal assistant, or chatbot and want to know whether your content is ready before you go live
- You’ve already deployed an AI tool and the answers it gives are inconsistent, incorrect, or incomplete
- You’re rebuilding or consolidating your knowledge base and want it structured for both human readers and AI retrieval from the start
- You’ve been told your documentation “isn’t good enough” for AI, but no one has told you specifically what that means or what to do about it
- You want an objective, external assessment rather than an internal team deciding their own content is fine
Find out what your AI is working with
Most teams find out their content isn’t RAG-ready after they’ve deployed their AI tool, not before.
A RAG Readiness Audit takes two to three weeks from scoping call to final report. It gives you a clear, actionable picture of what needs to change, before your AI tells your users something wrong.
Book a scoping call
The scoping call is free and takes 60 minutes. We’ll confirm which audit tier fits your situation and give you a written proposal within a few days.