How to build a reliable source of truth for AI agents

AI agents need a clear, up-to-date, and consistent source of truth to produce reliable content. When the documents they rely on are outdated, incomplete, or contradictory, even agents that follow instructions correctly produce inaccurate results.

The first time my AI agent wrote a comparison article, it described us using messaging we’d retired a year earlier. The model simply looked through our own files and copied what it found. Half the files were outdated, and two contradicted each other.

That explains nearly all the disappointing results I’ve seen from AI agents. While teams debate models and prompts, the real bottleneck lies in the reference material. Building a reliable source of truth requires accurate documentation, clear rules, examples of good output, and a way to test whether agents apply that information correctly.

10X your SEO with Semrush for Enterprise.

The world’s most powerful SEO platform, purpose-built for Enterprise.

Request demo

What a weak source of truth actually costs

If the reference is incorrect or absent, failures appear quickly. These are the ones I most often encounter in marketing work:

• The copy has a robotic, artificial quality: Since there are no negative examples to avoid and no actual customer language to imitate, all the content stays in the same dull tone — accurate yet forgettable and clearly written by a machine. Readers lose interest, and an editor has to start over, which takes longer than writing the text by hand.
• The positioning is inconsistent: When two documents contain two versions of the message, the agent switches back and forth between them: one page refers to you as a platform, the next calls you a tool, and the article sounds like three different companies wrote it.
• Inaccurate feature information: The agent offers a capability included in the roadmap deck but not in the product, omits one that was launched last month, or describes a feature as it functioned two releases earlier.
• Misunderstanding of competitors: The agent might find a rival’s price, but it can’t really grasp who you’re up against or where you shine, unless you spell it out. If your competitive landscape isn’t clearly documented, the agent will compare you to the wrong players or focus on irrelevant details. This is especially true for products like our own time tracker TMetric, where the market is crowded, competitors vary widely, and each ICP values different product features.
• Errors involving data and figures: The agent uses a metric based on last year’s definition or one no one has updated. As a result, the numbers no longer match across drafts. This erodes leadership’s trust and requires manual fact-checking of each item, wiping out the time the agent was meant to save.
• Missing SEO data: With no target keywords, no topic clusters, and no priority order, the agent won’t tailor the copy to your goals. It writes something generically reasonable rather than the page you actually need to rank for a target keyword.

None of these are your AI-model problems. Everything traces back to the source. Fix the source and the whole class of failure goes away.

Scope the agent before you feed it

Before you collect a single document, you need to clarify what your agent should do, read, and return, and whether it can make certain calls on its own or needs human approval. This filters everything else. 

The blog agent needs your editorial standards and your SEO rules, whereas the ad copy agent requires the list of claims it is legally allowed to make. If you give both the same large folder, each gets overloaded by the other’s context. The fastest way to ruin an agent is to give it more material to read.

Here’s a quick rule: If you can’t state in a single sentence which decision the agent is responsible for, it’s not ready for a source of truth. First, clarify its scope.

Give every fact one home and a date

When two documents contradict each other, which one takes precedence? If you can’t answer that clearly, neither can the agent. It may choose the wrong version and your messaging will drift. To avoid this, give each metric and claim a single official source. Point every other mention or reference back to that one place, so there’s no ambiguity about which is correct.

Give each fact a last-verified date and name an owner, a step most teams omit. An agent must regard a definition dated 11 months ago as different from one confirmed just last week, and it can only do so if the text includes the date. 

I keep the main definitions in a single file the agent references by ID number, so whenever a number changes, I update it once, and all subsequent drafts automatically use the new value. If you restate the same definition in five slide decks, you end up with five opportunities to contradict yourself.

Write for retrieval, not for reading

A brand book that wins design awards may still be useless to an agent. An agent may retrieve only the section of your 40-page PDF that’s relevant to a question. Write with that in mind. Write with that in mind. 

Make the sections short and independent (chunked), with each one covering a single idea. State the answer in the first sentence, as models tend to rely on the information that comes first. Use the question that the section answers as the title. Ensure the entire document is in plain text or Markdown format, not slide format, since that’s the AI-native format.

Try the simplest test: Ask the agent where it obtained a particular fact. If it quotes the exact line, then your system is working properly. If it hesitates or is vague, that means the fact is buried, and buried facts are gradually replaced by guesses.

Combine knowledge, examples, and testing

A reliable source of truth needs three layers: a knowledge base that defines the facts and rules, examples that show how to apply them, and a sandbox for testing the agent with real inputs.

They work together. Without examples, the agent produces technically correct but generic copy. Without rules, it can’t generalize to new requests. Without a sandbox, problems go undetected until publication. Most teams build the knowledge base and stop. That’s only one-third of the work — and the least important layer for ensuring quality.

Knowledge base: Define what’s true and how to communicate it

The knowledge base guides the agent’s claims and stylistic decisions. Include brand positioning, messaging pillars, the ideal customer profile and a persona for each buying role, tone of voice, editorial standards for each content type, keyword and search-intent rules, a glossary, and supporting evidence.

For the voice of the customer, use actual phrases from sales calls rather than summaries. For example, “we picked you because onboarding took a day, not a quarter” is more useful than a summary of 10 calls because it gives the agent the exact language customers use.

Examples: Show what good work looks like

Rules tell the agent what to do. Examples show how to do it.

The most useful example I’ve included is an editor’s tracked changes on an actual draft — a redline showing how an editor cut a three-sentence introduction to one line, replaced “solutions that empower teams” with the actual feature name, and moved a customer quote from the bottom of the page to the top.

The polished version shows the agent where to end up, while the redline shows how to get there. Also include one or two completed briefs, such as a how-to and a comparison listicle, two or three published articles that set the standard, and bad-to-good examples tagged by failure type, such as “too generic,” “buried the point,” or “no proof.” This helps the agent learn the rules rather than individual cases.

Sandbox: Test the agent with real inputs

The sandbox makes your source of truth readable and runnable. Include real sample data, a map showing which facts come from which systems, prompts, and human review points for each step.

For example, a Google Search Console export with one page that ranks and another that doesn’t helps the agent see the pattern it’s supposed to act on. Document where specific facts come from, such as pricing from the billing system and win-loss reasons from the CRM, so the agent retrieves information from the right source.

The knowledge base is table stakes. The examples and sandbox separate on-brand output from confident filler.

What to include in your source of truth

The full list I use for our source of truth covers the three layers and the guardrails that ensure integrity. Consider it a starting point: retain only what is necessary for your agent’s role and eliminate the rest.

Knowledge base	• Brand positioning, messaging pillars, and boilerplate • ICP and one persona per buying role • Tone of voice, plus a “how we sound vs. generic AI” note • Editorial standard with a spec per content type • Keyword and search-intent rules • Competitive set: who we compete with and where we win • Glossary and official terminology • Proof: case studies, results, testimonials • Voice of the customer: verbatim call and interview language
Examples to replicate	•  Filled-in briefs (landing pages, how-to, and comparison listicles) •  Two or three published pieces that set the bar •  Editor’s tracked-changes on real drafts •  Before/after pairs for content refreshes •  Bad-to-good pairs, each tagged with why it failed
Sandbox	•  Sample data with known-good outputs (e.g., a Search Console export) •  A map of which fact comes from which system •  Prompts and human review points for each step
Guardrails and upkeep	•  Exclusion lists: out of scope, needs sign-off, can’t keep current •  A last-verified date and an owner on each fact •  A version number and a review cycle

Say what the agent must not use

The information you tell the agent to leave alone is just as important as the content you include. Since the agent considers everything it can access to be fair game, it’ll use any item you don’t explicitly mark as off-limits. Keep a short, clear list of exclusions within the source of truth. Three such lists qualify for inclusion:

• Outside the scope: Any features, plans, or claims that aren’t yet live, since the agent will never produce a page for something that you can’t deliver. For example, a capability that is on next quarter’s roadmap.
• Approval is required: Publication needs the necessary legal or senior management approval, for example, for performance figures, security certifications, or references from specific customers.
• Up to date: Anything that can’t be reliably kept up to date doesn’t belong in your source of truth. This includes facts or data points that go stale quickly and aren’t actively maintained. A stale fact is more dangerous than a missing one because the agent treats outdated information with the same confidence as current information. If you can’t guarantee freshness, it’s better to remove that detail entirely rather than let it slip out of date and undermine your results.

Record the rules that are stored in people’s heads. Among my team, the greatest gap wasn’t a missing file. The keyword logic lived entirely in the memory of two strategists.

Instead of waiting for them to document it, we recorded a 30-minute conversation, transcribed it, and used the transcript as the rule document. Something rough and written is better than something polished and missing.

Rank the gaps by what breaks first

Conduct a gap analysis before involving an agent. List everything the agent needs. Mark each item as available, partial, or missing, note what’s lacking, and rank the items by how soon they’d cause a failure rather than how incomplete they are.

The hazardous items aren’t the ones that are missing; they are the partial ones, since they appear to be completed. Here’s the format I use:

Keyword and intent rules	Missing	Fix now	Lives in two strategists’ heads, with nothing to ground against
Gold-standard finished pieces	Missing	Fix now	No published examples for the model to match
Voice of customer	Partial	Fix now	10 sample calls, but agent needs the full corpus to generalize
“How not to sound like AI” set	Partial	Fix next	Guideline exists, missing tagged bad-to-good pairs

The pattern repeats every time. The knowledge base is usually fine. The gaps lie in the examples and the sandbox.

Here’s how to proceed: select 10 of the tasks that the agent got wrong and record the correct answer for each one. Each time you modify the source, perform the same 10 tasks. For each output, calculate the proportion of its statements that can be traced back to a particular line in the source. That figure is your grounding rate; it’s the single metric you should monitor. 

Whenever this rate falls, address the source rather than the prompt and add the missing example, remove the duplicate definition, or include the missing rule. Then rerun the 10 tasks and make sure the number returns to normal before you release the change.

Don’t evaluate the prose, correct one poor output by adding 10 new rules to the prompt, or construct your test set from tasks the agent already handles well. Base it on past failures rather than easy successes.

The marketer’s task

A source of truth fails when it’s written once and filed away. It functions when it’s scoped, dated, tested, and versioned just like any other product you release. Once you do that, the agent stops having to guess.

Each claim can be traced back to something you have approved, the figures agree in every draft, and the tone stays consistent without a human rewriting each one. That is how you end up with a content operation that doesn’t just use AI but one that runs on it.