We built our own knowledge base with vector and graph databases. Is it worth continuing, or should we buy?

Keep your knowledge base if it answers engineers' questions correctly today, one person owns it as their actual job, and it covers sources no vendor connects. Buy if the vector and graph databases now take more of that person's week to keep correct than to make better. The money and months you already spent are gone whichever way you choose, so they should not decide it. Upkeep load is the share of the team's time on the system spent keeping it correct (re-indexing, merging duplicate entities, syncing permissions, fixing wrong answers) rather than adding new questions it can answer. Our rule of thumb, not a measured threshold: when upkeep load passes half, the build has become a maintenance contract nobody signed. If you are still deciding whether to build at all, start with build or buy a context layer; this page is for teams that already built.

What vector plus graph gives you, and what it leaves to you#

The pairing is a sound design. The vector store finds passages that sound like the question. The graph stores relationships: this service calls that one, this engineer owns that module, this ticket led to that pull request. Together they answer questions that similarity search alone gets wrong.

Neither database knows which source is current or who is allowed to read it, so those are rules you write. The graph also costs more to build than people expect. Microsoft's GraphRAG docs open with the warning that "GraphRAG can consume a lot of LLM resources!" and suggest cheap models first, "before committing to a big indexing job" (GraphRAG docs). That cost comes back every time the code, the tickets and the wiki change, which in an engineering org is every day. A paper accepted to AAAI 2026 describes most GraphRAG systems as following a "build-then-reason paradigm" that "relies on a static, pre-constructed Knowledge Graph (KG)" (arXiv 2601.07192). A static graph goes out of date as fast as your org changes.

Signs it is worth continuing#

You can name the owner, and they are not doing it between feature tickets: they have a backlog and an evaluation set.

Answers carry citations, and engineers click them. If people check the sources and find them right, trust is built and worth keeping.

Your most important sources are unusual. A proprietary build system, an internal incident tool or a hardware test log will need a custom connector whatever you choose, so a system built around them has a real reason to exist.

The graph is small and stable. A few hundred services and owners that change monthly is a graph one person can keep accurate. For a fuller comparison of what each structure is good at, see context engine vs knowledge graph.

If all four hold, keep going and fund it as a product.

Signs it is time to stop#

The graph is drifting. Entity resolution is the work of deciding that two names refer to the same thing, and it never finishes. A talk at Neo4j's NODES AI 2026 conference in April puts it bluntly: "KGs decay — entities drift, duplicates accumulate, contradictions appear, and retrieval quality drops" (Neo4j NODES AI 2026). For example, "billing-svc", "payments" and "the billing service" become three nodes, and half the edges point at the wrong one.

Permissions are a filter you maintain by hand. Take Microsoft's security filter pattern for Azure AI Search, meant for indexes that can't use its built-in access lists. It says the user or group ID "is just a string" used in a filter. The same page says the service "doesn't provide APIs or features for populating the security field" (Microsoft Learn, updated August 2026). Slack, GitHub and Confluence each need their own sync job to keep that string correct. A missed sync means an agent quotes a private channel.

The answers are stale and nobody can say by how much. For example, an engineer asks how retries work and gets last quarter's answer. Our walkthrough of how a context engine weighs sources shows the rules that problem needs.

The builder has moved on, or wants to. We analyzed 194 conversations with engineering teams from May to August 2026. When a homegrown build hit a wall, it usually looked the same: the index quietly went stale while the maintainer got pulled onto roadmap work.

A one-month test before you decide#

Arguments about sunk cost go in circles, so measure three numbers for a month instead.

First, accuracy. Pick 25 questions from last month's real work, such as why a migration was rolled back or who owns the rate limiter, with answers you can check. Include some where the wiki and the code disagree, the case covered in which source the AI believes.

Second, upkeep load: have the owner log hours spent keeping it correct versus making it answer more.

Third, lag. Index lag is the time between a change landing in a source system and the knowledge base reflecting it. Merge a pull request that changes behavior and time how long until its answers change.

Then run the same 25 questions through a vendor trial on the same sources. Our buyer's checklist for evaluating a context engine lists what else to check. If your build matches it on accuracy with a low upkeep load, you have your answer. If it trails and upkeep is over half (the rule of thumb above), the next twelve months will look like this month.

What to keep if you switch#

Keep the 25-question test set and grow it; it is the most useful thing the project produced. Keep any connector for a source no vendor covers. Keep the graph schema as a written record of how your systems relate, because it tells you what questions people actually ask.

What you hand over is the upkeep. Unblocked is one option built for this. It is a context engine that continuously indexes code, pull requests, issues, docs, Slack or Teams and production tools, with access controls that mirror each source system (Unblocked docs). Engineers ask it in Slack or on the web; agents in Claude Code, Cursor, VS Code or JetBrains reach it through one MCP server. Wade Bruce, CTO at Fetch: "Unblocked brings everything together. You don't have to go digging through tools. It just works."

Run the test either way. A team that has built one of these understands the problem better than most buyers, and that makes the comparison fair.