Stop AI Slop From Contaminating Your Team's Knowledge Base

AI-linked vocabulary in GitHub pull request descriptions rose from 1.0% of words in January 2025 to about 45% in August 2026. That comes from one researcher's sample of 47,464 descriptions, published by Implicator in August 2026, so treat the exact figure loosely. The direction is not in doubt: most of what now lands in a repo's system of record has an agent's hand in it somewhere. To stop AI slop from contaminating your team's knowledge, you have to accept that the problem is not that agents write this content. The problem is that once it is written and indexed, retrieval cannot tell it apart from a decision a human made, so the next agent cites it as one. Unblocked's answer to this is built into how it ranks sources, which is the part most knowledge tools skip in favor of a label.

What follows: where the content enters, why retrieval can't distinguish it, whether the model-collapse research applies inside a company, and five approaches that deal with it.

Where does AI-generated content enter the knowledge base?#

Through the same front doors your senior engineers use. GitHub Copilot will write the pull request description for the person who opened the PR, and the docs tell you to review it carefully before you click Create. Most people don't. That description is now part of the repository's history, indexed by every tool that indexes PRs.

The other entry points look the same once you list them:

• Auto-generated documentation from docs-from-code tools and changelog generators, published straight into the wiki.
• Slack bot answers that get pinned, copied into a Confluence page, or quoted in a ticket.
• Tickets auto-closed with an agent-written resolution summary that nobody edited.
• Meeting-note and standup summarizers that produce a "decisions" section from a transcript.
• Code review bot comments that sit in the PR thread and get treated as review history.

Every one of these is indexed by the same pipelines that index a design doc written by the person who owns the system, with no distinguishing signal by default.

Agents are generating garbage that becomes a source for the next agent — how do we stop AI slop contaminating our knowledge?#

You stop it by treating provenance and authority as retrieval-time signals instead of hoping the content stays out of the index. The loop works like this. An agent answers a question by retrieving whatever is indexed. Suppose a previous agent's PR description, ticket summary, or Slack answer got indexed with no marker of where it came from. The next agent treats it as fact and builds a new artifact on top of it. That artifact gets indexed too.

Each hop loses a little: the PR description paraphrases the diff, the ticket summary paraphrases the PR description, and the Slack answer paraphrases the ticket. By the fourth hop, the agent is confidently citing a chain of paraphrases with no primary source anywhere in it. That is the hallmark of the model-collapse literature, applied to what enters a knowledge base rather than what trains a model.

No single fix stops this. The ones people reach for first (tag it, review it, filter it) reduce the volume without touching the moment the damage occurs, which is an agent citing a derived artifact as ground truth. The five approaches below are ordered by how close they get to that moment.

Why can't retrieval tell human decisions from agent output?#

Because similarity ranking embeds text and ranks by closeness to the query. It has no concept of who or what wrote a passage. A fluent, keyword-dense, agent-written PR description embeds at least as well as a terse human one, and often better, because agents write the way embedding models like to read.

There is also no provenance signal to rank on. Most systems don't store "generated by an agent" as metadata. When a source system does mark it (a bot account, an AI-generated flag on a summary), the mark rarely survives the first copy. Who authored the Confluence page you pasted the bot's answer into? You did. Who authored the ticket you summarized it into? Whoever closed the ticket.

Then satisfaction of search finishes the job. An agent stops at the first plausible hit. A wrong-but-fluent derived answer wins over a correct-but-terse primary one. This is the sibling of the problem where human-authored sources disagree with each other: there, the sources conflict; here, one of the sources should never have been a source.

There's drift between our repos and our documentation and agents can't tell — what tools handle this?#

Tools that rank sources by authority and recency instead of similarity alone. The freshness half of that answer is covered in depth in how to stop agents using stale documentation. That post walks through verification workflows, freshness metadata, docs-from-code generators, and authority-weighted retrieval. All four address drift between what the code does and what the docs say. If your problem is that rules files themselves disagree (a CLAUDE.md that contradicts an AGENTS.md), that's a different mechanism, covered in why coding agents can't pick a source of truth.

What neither of those pages covers is drift caused by agent-authored content re-entering the pipeline. A docs-from-code generator that regenerates the API reference nightly keeps that reference honest. It also produces a fresh, authoritative-looking document every night that says nothing about why the API looks that way. Freshness metadata will rank it first. That is a provenance problem sitting on top of a freshness problem, and the freshness fix makes it slightly worse.

Is AI slop really degrading agent output, or is that just a training-data problem?#

The measured research is about training, so let me be precise about what it shows. A September 2026 paper by Marchi, Silvestre, Gharesifard, and Tabuada (arXiv:2609.18878) uses the Fisher-Rao metric to derive bounds on how much fresh human data a model needs when it trains recursively on its own synthetic output. Below that rate, the model progressively loses the tail of the real distribution. A June 2026 paper from Qiao and colleagues (arXiv:2606.13732) adds a sharper point. When the people selecting training data have only a partial view, their filtering can accelerate collapse rather than prevent it. They keep what matches their local picture and discard the tail.

Neither paper measures collapse inside an enterprise retrieval pipeline. Nobody has published that yet, and I'm extrapolating. But the mechanism maps cleanly: a retrieval system that increasingly surfaces agent summaries of agent summaries is narrowing toward the same self-referential center, one hop away from training. The second paper's finding should worry anyone whose fix is a human curator with a partial view of the codebase, because that curator will keep the derived content that looks familiar. And the raw material is not scarce: at roughly 45% AI-linked vocabulary in the Implicator sample, PR descriptions with no agent involvement are now the minority in that corpus.

What are the approaches to stopping AI slop from contaminating retrieval?#

Five, and they are layers rather than competitors. Three of them reduce how much derived content reaches the index. One holds it at the door. Only one acts when an agent is about to cite it.

Two notes on the first row. Confluence's own automation actions can add a label when a page is published, so tagging pages that a bot account publishes is a ten-minute rule to write. The regulatory push behind marks like that is real. The EU AI Act's Article 50 applies from 2 August 2026 and requires providers of systems that generate text to mark outputs in a machine-readable format and make them detectable as AI-generated. Systems already on the market get a grace period to 2 December 2026. The accompanying Code of Practice was finalized in June 2026, and C2PA content credentials give you the general-purpose format. The practical limit hasn't moved: a mark on the output does nothing once a person pastes that output into a page under their own name.

On the second row, Guru's verification workflow is the cleanest implementation of human trust flags. For this post's purposes, its value is that a verified card was blessed by a named person, which is exactly the signal a pasted bot answer lacks. The blind spot is the same one the June 2026 paper describes. A verifier with a partial view will bless what looks right to them.

Where does Unblocked fit among these five approaches?#

In the third and fifth rows, which is why it can stop the loop at the retrieval moment rather than upstream of it. Unblocked builds an expert graph from pull request and review history, so authority comes from what shipped and who was trusted to review it, not from who typed fastest. It treats code as ground truth for current behavior and reads documents and summaries as statements of intent. When a merged PR and a Slack bot's paraphrase of that PR both match a query, the PR wins. When two sources genuinely conflict, the engine surfaces the conflict with a stated reason instead of silently discarding one side. The ranking of primary sources above derived artifacts is part of the engine itself rather than a filter bolted on afterward. That difference is what customers describe when they talk about trusting the output:

I used to tolerate AI tools even when they missed the mark, but Unblocked has been consistently dependable. Every piece of feedback I've seen so far has been actionable. My default is to take its recommendations straight into Cursor because they almost always point to a fix I need to make.

Cody Robinson — Development Manager, Clio

The same ranking is what teams at Clio, Drata, and RB Global rely on when Unblocked reviews their code, and what made it Clio's default reviewer: a reviewer has to know which prior artifacts to believe before it can say anything useful about a new diff. If your concern is the code side of this, verifying AI-written code against team conventions covers that lane.

Frequently asked questions#

What counts as "AI slop" in an internal knowledge base?#

Any agent-generated artifact that gets indexed and retrieved without a signal distinguishing it from a human decision. It is not a judgment about writing quality. A June 2026 Columbia SIPA report notes there is no consensus definition even for public slop, and describes it as high-volume synthetic content optimized for engagement over depth. Inside a company, the useful definition is narrower: derived content that retrieval treats as primary.

Can you just detect AI-generated text and filter it out?#

No, and it's the wrong axis anyway. Stylistic detection is unreliable, gets worse as models improve, and would also filter the human who writes like a model. The fix is provenance and authority at retrieval time, as described in the section on why retrieval can't tell human decisions from agent output above. The useful question is what a passage was derived from.

Does RAG suffer from model collapse the same way model training does?#

The mechanism is analogous: self-referential content narrows the signal toward a center and loses the tail. The 2026 model-collapse papers cited above measure this in recursive training. No published study yet measures collapse inside a company's RAG pipeline. Treat the analogy as a well-grounded extrapolation rather than a proven equivalence.

Should AI-generated PR descriptions and auto-closed tickets be trusted as sources?#

Conditionally. They are useful when retrieval weights them as derived from something else (the diff, the actual resolution) and lets that primary source win when both are available. They are dangerous when a system treats them as independent sources of truth, which is the default for anything that ranks by similarity. The question to ask of your tooling is whether it knows the difference.

So what actually stops the feedback loop?#

Tagging, verification, and quarantining all reduce the volume of derived content that reaches the index, and you should do the cheap versions of all three. None of them acts at the point where harm occurs. That point is the moment an agent cites something as ground truth. The only approach that operates there is retrieval that ranks primary sources (code, merged PRs, the humans who made the decision) above content derived from them, by design. Provenance marks won't survive the copy-paste, and reviewers can't keep up with commits.

The architecture-level version of this argument, including how documents are treated as one signal among several and cross-referenced against code, is in Inside the Unblocked context engine. If you want to see what your agents are actually citing today, connect your sources to Unblocked and ask it a question you already know the answer to. The sources it shows you are the ones your agents are trusting.