There is drift between our repos and our documentation and agents cannot tell. What handles this?
No single tool stops documentation drift, so handle it in three places. Catch it in the pull request by making doc owners review code changes. Detect old drift with scheduled checks that test docs against code. Then put a context engine between the agent and the docs, so answers weigh the docs against the current code and show the conflict.
No single tool handles documentation drift, because drift is a gap that widens a little with every merged change. What handles it is three checks in three places: one in the pull request, one on a schedule, and one at the moment the agent asks. Documentation drift is the gap that opens when code changes merge and the docs that describe that code do not change with them. The first two checks shrink the gap. The third, a context engine that compares the docs against the current code before it answers, is what keeps agents safe from the drift you will never finish fixing.
Why agents cannot tell the docs are stale#
An agent has no sense of time unless a source supplies one, so a README, a Confluence page and a function all look equally authoritative. Say your onboarding page and the repo README both tell new engineers to ship with make deploy. In March the team moved deploys to a GitHub Actions workflow and deleted the old target, but nobody updated either page. Ask an agent how to deploy and it will quote the documentation with confidence, because two sources agree and the code says nothing about deployment in plain English.
We analyzed the questions buyers asked on our sales calls from May to October 2026, and six companies asked some version of this one. Drift is two or more records about the same fact, written at different times, with no link between them. Why retrieval tends to pick the wiki over the code is covered in our wiki says one thing and the code does another. This page is about what to put in place so the gap stays small and the agent can see it.
Catch drift in the pull request#
The cheapest place to stop drift is the moment it starts. If the docs live in the repo, you can make the people who own them review changes to the code they describe. Code owners are the people or teams a repository names as responsible for a set of files. On GitHub, code owners are requested for review automatically when a pull request changes files they own, and a branch rule can require "review from Code Owners" before a merge (GitHub Docs).
In practice that means three small habits:
- Keep how-to docs, runbooks and agent instruction files in the same repo as the code they describe.
- List the docs owner as a code owner for the code paths those docs cover, such as the deploy workflow folder.
- Add one line to the pull request template: "Which doc does this change make wrong?"
This catches a lot, but not documentation that lives outside the repository, like Confluence pages and Notion runbooks. It also depends on a busy reviewer noticing, which is why you need the next two checks.
Detect the drift that already happened#
Most teams already carry years of drift and need a way to find it without reading every page. Two kinds of automated checks help.
The first tests docs against code. A study accepted at FSE 2026 describes CASCADE. It uses large language models "to generate unit tests directly from natural-language documentation". It only reports a mismatch when the real code fails a test that code written from the docs passes. That rule exists because, in the authors' words, "false positives carry serious consequences" for the people who must confirm each report. Run on open source projects, it found 13 mismatches nobody knew about, and 10 have since been fixed (arXiv). This works best on API docs and docstrings, where the claim is precise enough to test.
The second checks the files agents read first. Claude Code treats CLAUDE.md as context, not enforced configuration, and its docs warn that "if two instructions contradict each other, Claude may pick one arbitrarily". It ships a /doctor prompt-audit command that looks for "references to files or commands that don't exist, and files that contradict each other" (Claude Code docs). It reads instruction files such as CLAUDE.md and AGENTS.md, not the README, so if your CLAUDE.md also says make deploy, it should flag that line. Run it monthly. Why these files go stale so fast is the subject of rules-file rot.
Check every answer against the current code#
The first two checks reduce drift but never bring it to zero, because code moves faster than any review cycle. Someone also has to decide which record wins when two disagree, a question I took apart in who decides what is true for AI agents. So the last check has to happen at the moment the agent asks.
A context engine is a service that indexes code, pull requests, tickets, docs and chat together and answers from all of them at once. Unblocked is one: it connects to sources such as GitHub, Confluence, Jira and Slack, and its documentation says it "reconciles information across sources, including sources that contradict each other". Its engine treats code as the strongest evidence of current behavior, and when a conflict cannot be cleanly settled it presents both sources rather than dropping one. Coding agents reach it over MCP, and access mirrors the permissions in each source system. In the deploy example, the answer would name the GitHub Actions workflow, cite the March pull request that removed the old target, and point out that the README and the onboarding page still describe the old way.
That last part matters as much as the correct answer, because a cited conflict tells the engineer which doc to fix and so feeds the other two checks. The four common ways tools handle stale docs, from verification badges to docs built from code, are compared in tools that stop AI agents acting on stale documentation.
Where to start this quarter#
Here are the four checks side by side.
| Check | When it runs | What it catches | What it misses |
|---|---|---|---|
| Code owner review | Every pull request | Docs in the repo that the change makes wrong | Wiki pages outside the repo; tired reviewers |
| Docs-against-code tests | On a schedule or in CI | Precise claims in API docs and docstrings | Prose runbooks and design notes |
| Instruction-file audit | Monthly | Dead commands, missing files, contradictions | Facts that are wrong but still parse |
| Context engine | Every question | Conflicts between any sources, shown with citations | Nothing gets fixed unless someone acts on it |
Start with the documentation agents consult most often and the code that changes most frequently. For most teams that means the agent instruction files, the deploy and release runbooks, and the API docs for services under active work.
- Move those docs into the repo where you can, and add code owners for them.
- Add the "which doc does this make wrong?" line to the pull request template.
- Run an instruction-file audit and fix what it finds.
- Put a context engine in front of your agents, so the drift that remains is shown with its sources instead of passed on as fact.
The order matters less than the last step. Drift will always exist; what you control is whether your agents can see it.
