All answers

Should we build or buy a context layer for our AI coding agents?

Dennis PilarinosDennis Pilarinos·Oct 27, 2026·Context Engine · Comparisons

Build if the whole problem is one repo and two or three sources, or a platform team wants to own ingestion, permissions and evaluation as a product. Buy if knowledge sits in five or more systems and agents need a reconciled answer rather than search results. The first month of a build is cheap. Months two through twelve are the cost, and they are what to budget for.

Build a context layer if one repository and two or three sources are the whole problem, or if a platform team wants to own it as a product and has the headcount to keep owning it in month nine. Buy one if your engineering knowledge is spread across five or more systems with different permission models and your agents need one reconciled answer rather than a list of search hits. A context layer is the service between a coding agent and the places where a company's knowledge lives (code, pull requests, tickets, docs, chat); it returns the relevant, permitted, current slice on request. I run a company that sells one, so read what follows with that in mind.

When building is the right call#

One repository, a few sources. If the agent needs the code, the README and one ticket tracker, a rules file plus a small retrieval script covers it.

A small team where one person holds the whole map, and ten engineers in the same channels already know which document is current.

A platform team that wants to own it, with a backlog, an owner and an evaluation suite. That is the only version of the build that survives the first year. Octopus Deploy's 2026 Future of Platform Engineering report, written up in The New Stack from 379 platform professionals, found teams building mostly from scratch about 3.5 times more likely to report a lack of clear strategy as an obstacle. That was 57.9 percent, against 15.6 percent for teams building on a template. Its exceptions matter: teams build where a feature is "bound to a specific tech stack and golden paths that off-the-shelf tools rarely provide," as the article puts it.

Unusual sources, such as a proprietary ticketing system, lab notebooks or a mainframe change log. No vendor connects those on day one. You will build that connector regardless; if those sources hold most of the knowledge, build the layer around them.

What the build is in month one#

Month one is the easy part, and it is what persuades teams the whole thing is easy. Chunk the wiki, embed it into a vector store, retrieve per query, expose it over MCP. One engineer does it in a few weeks alongside their real job, the demo lands, and adjacent teams ask for access. We analyzed 194 conversations with engineering teams between May and August 2026 (the homegrown context stacks data). Just over four in ten had built something like this. Roughly three in four of the builders described the same arc: build, early success, a staleness and maintenance wall, then a search for alternatives. We met them at the last step; apply that selection bias to every number. "AI removes the effort of writing code, not the work of owning it," as The New Stack piece puts it.

What the build is in months two through twelve#

This is the budget line; each item is a job someone will hold.

Connectors and their auth. Every source has its own API, token lifecycle and rate limit, and each one changes under you. GitHub's REST API gives a user token 5,000 requests per hour. A GitHub App installation starts at the same 5,000, gains 50 per hour for each repository above 20, and caps at 12,500. Do the arithmetic for a thousand repositories. Atlassian prices Jira reads in points for Forge, Connect and OAuth apps, against a shared pool of 65,000 points per hour across all tenants; identity and access reads (users, groups, permissions) cost two points per object. Those are the calls the next item needs.

Permission mirroring across every source. Permission mirroring is keeping the layer's view of who may see what in step with the source systems, so an engineer's agent never surfaces a Slack channel or repository the engineer cannot open. GitHub teams, Jira project roles, Confluence spaces and Slack private channels are four different models. A May 2026 paper on production RAG names "data staleness, tenant data leakage, and query composition explosion" as its three root causes and traces them to "the conventional split-system data layer" (arXiv 2605.03275). Leakage in an internal tool is a security incident, so this item needs a named owner.

Re-indexing. Documents change, people move teams, channels get archived. A nightly full re-index is the simplest design and the one the rate limits above rule out at scale, so you build incremental sync per source.

Conflict handling. Conflict handling is the rule set that decides which source wins when the Confluence page, the merged pull request and last month's Slack thread disagree, and that can say why. Similarity search has no opinion; you write the recency and authority rules yourself.

Evaluation. Stack Overflow's 2026 survey found 48 percent of developers trust AI "when they can easily validate the answers" (Stack Overflow Developer Survey 2026), so the layer needs citations and a test set from the start. The LayerRAG-Bench benchmark from July 2026 found that normalizing schemas took schema-drift success from 0.000 to 0.913, while "stale evidence, missing tool output, denied permissions, and wrong-session context are not recovered" (arXiv 2607.27353). Grading on groundedness alone produced false positives under stale evidence. Each failure class needs its own test.

On-call. Someone gets paged when the Slack token expires at 2 a.m. and every agent in the company starts answering from a six-day-old index. That piece cites David Tuite's "build-versus-buy trap," with teams sinking "6 to 18 months into upgrades and operational overhead instead of shipping value" on custom interfaces.

When buying is the right call#

Buy when the sources are many, the permission models differ, and nobody wants the pager for a system that is not the product you sell. Unblocked is built for that case, and other vendors sell here too. It connects GitHub, GitLab, Slack, Teams, Confluence, Jira, Linear, SharePoint, Datadog and Sentry, keeps the index current, applies access controls that mirror the source systems, and answers through one MCP server (docs). Alec Robins, CTO and co-founder of Rally, put it this way: "It wasn't worth our time to cobble that together internally. And going to market, the enterprise solutions didn't make sense for our size of company. For what we needed, there really wasn't anything that solved this except for Unblocked."

The longer argument, connectors versus a reconciled answer, is in A pile of MCP connectors is not a context engine.

A fair way to decide#

Write down three things before anyone writes code:

  • The list of sources and the permission model each one uses.
  • Who will own ingestion, permissions and evaluation in month seven, and what they will stop doing to make room.
  • Twenty questions with known answers from your own history, for example a schema decision that was reversed, half of them with a wrong document in the way.

Run the twenty against the month-one build and against a vendor trial. If the build wins, keep it and fund the owner. If it ties, the twelve-month list above is the tiebreaker. The checklist we use when teams evaluate us is in how to evaluate a context engine.