# How does an AI knowledge tool decide between a recent Slack message and a year-old Confluence doc?


URL: https://getunblocked.com/answers/how-an-ai-knowledge-tool-weighs-a-recent-slack-message-against-an-old-confluence-doc/
Published: 2026-10-30T17:00:00Z
Updated: 2026-10-28T00:01:33Z
Author: Brandon Waselnuk
Categories: Context Engine, Engineering Insights

**Short answer:** Most tools do not decide. Enterprise search ranks both by how well they match the query, nudged by recency and activity, and hands the agent whichever scores higher. A context engine weighs them: it checks the merged code, the authors, the date, whether the doc was superseded and who else cited it, then returns the current fact with both sources dated and the conflict shown.

---
In brief, most AI knowledge tools do not decide between a recent Slack message and a year-old Confluence doc. They rank both by how well each matches the question, with a nudge for recency and activity, and pass the higher scorer to the model, which treats whatever it was handed as true. **A context engine** is a service that indexes code, PRs, chat, tickets and docs together and weighs the two. It checks what the merged code says, who wrote each source and whether that person owns the area, when each was written, whether the doc was superseded, and what else cites it. **Reconciliation** is that weighing step, done before the answer is written, with both sources dated and the disagreement shown.

## The signals a tool can use

We analyzed our sales calls from May to October 2026 and found that eight companies asked some version of this question. For example, picture a Slack message from last Tuesday saying the nightly reconciliation job moved to 02:00 UTC on the new scheduler, and a Confluence runbook from last October that still says 23:00. Seven signals separate a guess from an answer.

- Recency. The Slack message is newer, which proves little on its own; it could be a wrong guess.
- Source type. A runbook is written to be read later; a chat message is written for the people in the thread.
- Authorship and ownership. Does the poster own the scheduler? Is the runbook author still on the team?
- Whether the code agrees. The scheduler config says one time or the other, and the merged commit has a date and reviewers.
- Whether the doc was superseded. Did a PR or a newer page replace the runbook, even if nobody edited it?
- Who else cited it. Later threads, tickets and PRs that point at one source, not the other.
- Permissions. Whether the asker may see each source at all.

Search reads recency, permissions and page views. A context engine reads all seven.

## Why enterprise search ranks instead of reconciling

**Enterprise search** is a product that indexes every app a company uses and returns ranked, permission-filtered matches. It was built to find documents, so the signals it reads are the ones that sort a list. Glean's code search docs say ranking "adds popularity, recency, and affinity signals to prefer the code people actually use" ([Glean docs](https://docs.glean.com/security/how-code-search-works), updated September 2026). Confluence Data Center's search docs weight recently updated items higher because fresh content "is likely to be more relevant than idle or older content", with the lowest recency weight for anything older than a year ([Atlassian](https://confluence.atlassian.com/conf101/ranking-of-search-results-1652925520.html)). Rovo answers come "along with any relevant sources", drawn only from content the user can access ([Atlassian Support](https://support.atlassian.com/rovo/docs/search/)). Slack's AI search uses "in-house ranking models" and "will never surface any results that Slack's regular search would not" ([Slack](https://slack.com/help/articles/28310650165907-Security-for-AI-features-in-Slack)).

Each is a fair description of good search. None is a decision. In the pages we read, no vendor states what happens when a Slack message and a Confluence page disagree about the same fact. The Slack message may win for being recent, or the runbook for having more views. Either way the model gets one, or both unlabelled, and writes a confident sentence. A May 2026 paper names the flaw: retrieval-augmented systems "implicitly assume mutual consistency among retrieved documents" ([arXiv 2605.17301](https://arxiv.org/abs/2605.17301)). That assumption breaks the moment the wiki ages.

## What a context engine does with the same two sources

A context engine starts from a different question: not "which document matches best" but "what is true today, and how do I know". For Unblocked, merged code is "the strongest ground truth for how the system behaves right now", so the scheduler config and its merge date are read first. Results are then "ranked and related using recency, authority, the expert graph" and the asker's own area of work. The expert graph is built "primarily from pull request activity and commit history", and review counts as well as authorship, so it can tell whether the Slack poster owns the scheduler. "Recency matters, especially for stream-of-consciousness sources like Slack", and a recent statement from the identified expert can override an older document. When the signals genuinely conflict, "the engine says so rather than picking a side and hiding the disagreement" ([inside the Unblocked context engine](https://getunblocked.com/blog/inside-the-unblocked-context-engine/)).

The docs say it "reconciles information across sources, including sources that contradict each other" ([Unblocked docs](https://docs.getunblocked.com/what-is-unblocked.md)), and each answer links the files and pages it rested on ([Unblocked docs](https://docs.getunblocked.com/unblocked-chat/cited-sources.md)). The engine maps permissions from the source systems and enforces them at query time, so it weighs the runbook only for people who can open it.

## The same weighing during an incident

Nazmus Sakib, a software engineer at Workday, saw the effect during on-call incidents. The pattern match he mentions across old Slack threads and Confluence pages is the weighing above, and the tool does it, so the engineer reads a result instead of running the search. [A context layer is not a context engine](https://getunblocked.com/blog/a-context-layer-is-not-a-context-engine/) draws the line between indexing sources and reconciling them. In his words: "When I'm on call and we hit a data discrepancy, that investigation used to take a full day. Now the agent calls Unblocked to pattern match against past Slack conversations and Confluence pages — and I have an answer in 30 minutes. Teammates without Unblocked licenses have told me the results are more useful and easier to work with than what they're using instead."

## What a good answer looks like

Consider the job above. A good answer reads: "The nightly job runs at 02:00 UTC, set in scheduler.yaml and merged 14 October with two reviewers. The runbook says 23:00, was last edited October 2025, and predates the scheduler migration. The Slack message from 21 October matches the code." Four things make that answer trustworthy.

1. It names the current fact and the artifact that proves it, with a date.
2. It dates every claim, including the stale one, so you can see the gap.
3. It shows the conflict instead of hiding it, and tells you which source governs.
4. It cites both sources with working links, so someone can fix the runbook.

A September 2026 study built 317 verified knowledge reversals, then fed models the stale document. With no instruction to trust it, stale retrieval flipped 30% of correct Llama answers and 37% of Qwen answers, and telling the model to follow the document raised that to 66% and 75% ([arXiv 2609.31342](https://arxiv.org/abs/2609.31342)). Told when the old evidence expired, larger models switched almost perfectly; the authors conclude that "Reliable RAG therefore requires selective trust". That is what the seven signals buy: a dated, owned, code-checked source beats a well-ranked one. If your tool returns a list, the weighing is your job. A sibling answer covers [which source an agent believes when the wiki and the code disagree](https://getunblocked.com/answers/wiki-says-one-thing-code-does-another-which-does-ai-believe/) and what to do with the stale page. The [best AI documentation tools for engineering teams](https://getunblocked.com/blog/best-ai-documentation-tools-engineering-teams/) roundup compares tools that keep runbooks current.

## Read more

- [Inside the Unblocked context engine: how we decide what your agent needs to know](https://getunblocked.com/blog/inside-the-unblocked-context-engine/)
- [A context layer is not a context engine](https://getunblocked.com/blog/a-context-layer-is-not-a-context-engine/)

## Related answers

- [There is drift between our repos and our documentation and agents cannot tell. What handles this?](https://getunblocked.com/answers/documentation-drift-between-repos-and-docs-for-ai-agents/)
- [How do I mark old Confluence pages as deprecated so AI stops using them?](https://getunblocked.com/answers/mark-confluence-pages-deprecated-for-ai/)
- [MCP tools are eating the whole context window in Claude Code. How do other teams give agents company context?](https://getunblocked.com/answers/mcp-tools-filling-claude-code-context-window/)
- [We have context all over the place and constantly being generated. How do we organize it so AI pulls the right thing?](https://getunblocked.com/answers/organize-scattered-context-so-ai-pulls-the-right-thing/)