Why AI Assistants Fail on Context in the Enterprise

The four previous posts describe four different assistants: creating, transforming, checking, locating. Reading back through them, I noticed that all four end with the same limitation. The transformer is only as good as what is connected. The critic fundamentally cannot find duplicates without the full inventory context. The tracker stands or falls with logs and traces. The creator has to declare assumptions because something is missing.

Four posts, one finding. This post is about how I bring the missing knowledge to the model — by hand at first, later via MCP — and why connecting it is not enough on its own.

The Symptom Looks Like a Prompt Problem

The sequence was the same every time. An assistant delivers a result that looks sound: all fields filled, clean language, correct structure. You read it, you nod — and only later do you stumble over the fact that it answered the wrong question.

Because the questions a result hangs on in an enterprise are not aimed at the text but at the inventory:

• Does this requirement contradict an existing function?
• Does it extend one — and which?
• Does it perhaps already exist?

None of these three questions can be answered from the artifact at hand. And the reflex, when the answer is wrong, is nonetheless: work on the prompt.

I did that for a while. The results genuinely did get better — more precisely phrased, more complete, more consistent in structure. Only nothing changed about those three questions, and in hindsight it is clear why:

Prompt work improves phrasing and form. It cannot replace knowledge that is not in the room.

The uncomfortable part is how the model deals with the gap. It does not say "I do not know that". It fills it with something plausible. And in a requirement, plausible is indistinguishable from correct — exactly the point where the creator carries the obligation to declare assumptions, and the transformer the obligation to attach an origin to every enriched statement.

Knowledge the Model Could Not Have

In parallel with the toolkit, fairly early in its development, I moved a framework to a new version. A completely different task: no product process, no requirements, but code.

And there I faced the same gap, just from a different direction. The model knew the framework — but not the version I wanted to move to. It was newer than the model's training data: no release notes, no migration guides, no known issues. A model without that information can only suggest what fits the state it knows. And that sounds just as convincing as the right answer.

The workaround was unspectacular. I put the release notes, the migration guide and the known issues into the project as local files, for exactly this update process, and referenced them in the prompt — so that the model would draw on them for this task.

Two things belonged together: the files and the reference to them. Information sitting somewhere in the project does not mean the agent reads it for the task at hand. The reference in the prompt was the signpost. The whole thing was manual work — assembled for one task, maintained by hand, back to square one for the next project. But it worked.

Two completely different tasks, two different gaps:

• Public knowledge that is newer than the model — new versions, release notes, known issues.
• Internal knowledge that was never public — the inventory: solution concepts, existing functions, earlier decisions.

And the same root cause: the model lacks something no prompt can replace. That was the point where I stopped treating it as a property of the individual assistant and started putting a question in front of everything:

If a task requires knowledge the model does not have, every prompt round is wasted time as long as that knowledge is not connected.

That sounds banal. In practice it is the rule that has saved me the most afternoons — because it stops a kind of work that feels very productive and changes nothing about the actual bottleneck.

Extended Context — and the Question of Which Source for What

Some time later the Model Context Protocol arrived, MCP for short. With it I no longer had to collect knowledge as files for a single task. Instead, the services where that knowledge already lives can be connected cleanly: a ticket system such as Jira, a knowledge base such as Confluence, monitoring tools for logs, metrics and traces. That extends an agent's context with what the company knows. What such a connection looks like is described in End-to-End Development with Agents and MCP. MCP servers only made it into the toolkit considerably later.

That solves the manual work. It does not solve the signpost question — it makes it bigger. An agent with several connected services has to decide for every task where to look. And every server brings its tool descriptions into the context, whether the task needs them or not.

What the reference in the prompt used to do is now the job of the AGENTS.md. Of all things, this file has become contested. A study from ETH Zurich concluded in early 2026 that such context files do not generally improve the success rate of coding agents, while increasing inference cost by more than 20 percent on average. Repository overviews turned out not to help there — instructions, on the other hand, are followed reliably.

In the toolkit, the AGENTS.md was nonetheless not optional but essential. The difference lies in the starting point. The study measures tasks within a single repository — there, an agent can orient itself by searching and reading. The toolkit has more than a hundred repositories connected. No agent skims those quickly to get an overview. Working in one repository, you find your way around. Working in a landscape of a hundred services, you need a map first.

That map is the AGENTS.md — explicitly not documentation, but rough guidance. It answers three questions:

• Which service is responsible for what? One line per service, no more.
• Which source fits which task — and which does not?
• Where do the solution concepts live? When an architecture question comes up, the agent knows where to look.

Simplified, for illustration:

## Map

Rough responsibilities. Details live in the respective repository.

- Check service: business validations and plausibility checks
- Product service: product data, variants, prices
- Payment service: payment methods, billing, refunds
- Voucher service: vouchers, discount codes, redemption
- …

## Which source for which task

- Working out a requirement: ticket system for the item and its linked
  items, knowledge base for solution concepts and the glossary.
  Do not use monitoring.
- Analysing a bug: first the logs for the error message, then the traces
  of the affected call chain.
- Architecture questions: look in the solution concepts in the
  knowledge base.
- If a required source is unreachable: say so in the result instead of
  carrying on without it.

The altitude is what matters. The file does not describe how a service works, only what it is there for. The agent finds the rest itself — once it knows which of the hundred repositories to search. That even matches the recommendation of the same study: a context file should only contain what goes beyond what is already in the code. A responsibility map across a hundred repositories is in none of them.

With the file, the results were substantially better. I have not measured that — more on this under the limits — but it is the reason I do not transfer the study's result to this case.

There is a second reason the file stays deliberately short. I learned it in the toolkit — the uncomfortable way.

More Context Is Not Better Context

In the toolkit, context grew in three stages, each because the previous one made its limit visible:

Stage 1 went surprisingly far — for everything to do with form. Stage 2 cost the least and delivered the most per unit of effort: an artifact references its neighbours anyway, and following those references mechanically is cheap. Stage 3 is the expensive one and the only one that really answers the three questions above. It is where I failed first.

Problem 1: more sources, worse results. Once stage 3 existed, the obvious configuration was right there: every assistant gets access to everything. Why would you withhold anything?

The result was worse than before. Not dramatically wrong — blurrier. An assistant that was supposed to phrase a requirement suddenly pulled in passages from an operations manual, because the same terms appeared there. The results were longer, more cautious, more general. And the review time went up, because now you had to wonder about every paragraph where it actually came from.

What helped was defining sources per assistant type rather than globally. The creator sees different sources from the critic, and the tracker sees operational data that has no business being in a requirement. That is the same thought as with the AGENTS.md and the output schema: the model should not decide what is relevant; the person who knows the task should.

That matches something I wrote about overloaded agents in the agentic coding series: a longer context window is not a better one. What is carried along without being needed is not neutral — it competes for attention.

Problem 2: the duplicate check found similarity, not sameness. Even with the inventory connected, the assistant reliably finds similarly phrased requirements — but not reliably the same requirement in different words. And two requirements that sound almost identical can be functionally different. The consequence was not a better search index but a more honest output: the assistant states what it checked against and what it did not. Why a check result without a stated scope is an all-clear nobody can back up is covered in the critic.

To Take Away

From both cases and the toolkit, four points remain for me:

1. Name the gap first, then prompt. If the model lacks knowledge — because it is newer than its training data or because it was never public — no prompt round will rescue the result.
2. Put the knowledge where the agent will find it. For a single task, local files in the project are enough. For knowledge that keeps being created — tickets, documentation, logs — MCP is the clean way.
3. Connecting without a signpost is half the work. The reference in the prompt, the AGENTS.md, the source configuration in the toolkit: it is the same job every time — telling the agent which source counts for which task. And the signpost stays rough: what something is responsible for, not how it works.
4. What is not connected belongs in the result. A known blind spot that is named in the result is information. The same blind spot, unmentioned, is a silent false statement.

The fourth point is the one I underestimated the longest.

Limits

Permissions are part of the problem, not a detail afterwards. An assistant with inventory access inherits an access question: who may see which source, and what happens when a result contains knowledge the recipient should not see? On a small scale that is not a question; in an enterprise it is the one that decides the rollout.

The inventory is not the truth. Connected documentation can be stale, and an assistant then quotes it correctly and is still wrong. That is not a model error — it is the same error a new colleague would make. The only protection I have is the origin reference: whoever sees the source can check it.

Diagrams stay out. Solution concepts often carry their decisive information in a picture, and the system does not read them. Why I deliberately decided against it is on the project page — in short: the cost structure changes for every document, not just for the few where it helps.

And no robust impact measurement. What I can offer is feedback: since the context was extended, users report that the results have got better — that feedback did not come before. That is worth something and it is not a measured value. Why I still do not claim a percentage is in my post on measuring impact.

Conclusion

The toolkit taught me one thing I would have assessed differently beforehand. I started with the assumption that the hard parts are the task and the output schema — that is, what the four previous posts are about. Both matter, both were work, and both were solvable.

The bottleneck was neither. It was access to knowledge the model does not have: to what the company already knows — and sometimes simply to what is newer than the model.

That is good news insofar as it is not an AI problem. Whether a requirement connects to what exists depends on somebody knowing the inventory — that was true before as well; it was just carried by an experienced human, and nobody wrote it down as a property of the system. An assistant merely makes that dependency visible, because it cannot substitute experience for it.

The rule I take away is therefore less technical than the topic sounds: before I make an assistant better, I check whether it can see what it would need to know — and whether it knows where to look. And where it cannot, it says so — instead of substituting for it.

→ The toolkit at a glance