Context engineering for codebases: test relevance, cost, and failure rate

context engineering coding agents shown as an experiment dashboard comparing broad and routed context packages

Last updated September 18, 2026

4 min read

A fictional platform team loads 42 instruction files into every agent session. The model reads more and costs more, yet misses the two rules tied to the changed module. The failure is not lack of context. It is poor selection and weak evidence about what was relevant. This is an explicitly fictional composite, not a customer result.

In this article
  1. Context is a selection problem
  2. Run an on-off repository experiment
  3. Record retrieval evidence beside output
  4. Prefer routed context over one permanent dump
  5. Frequently Asked Questions
  6. What next?

Context is a selection problem

Anthropic’s engineering guidance treats context as finite and recommends keeping the highest-signal information available. For codebases, the hard part is not only compression. It is selecting the right module rules, architecture decisions, interfaces, tests, and current documentation for the stage.

Anthropic on context engineering bounds this claim. Vendor engineering guidance, not comparative outcome research.

Evidence pointInterpretation
142 files loaded
22 files relevant
320 paired tasks
45 outcome measures

Treat the table as a working diagnostic. Replace illustrative labels with repository evidence before making a budget, staffing, or release decision.

Run an on-off repository experiment

Choose 20 representative tasks. Run each with a broad default package and a routed package under the same model and tool settings. Measure tokens sent, relevant sources retrieved, task completion, correction turns, and the failure class. Report limitations instead of declaring a universal winner.

1Select representative tasks. Record the artifact, owner, and completion evidence before moving on.
2Freeze tool settings. Record the artifact, owner, and completion evidence before moving on.
3Compare broad and routed context. Record the artifact, owner, and completion evidence before moving on.
4Review failures and retrieval receipts. Record the artifact, owner, and completion evidence before moving on.

Order matters because later checks inherit the intent and scope established earlier. A skipped step does not remove uncertainty. It hands that uncertainty to a reviewer under time pressure.

Record retrieval evidence beside output

Every retrieved slice should carry source path, revision, reason for inclusion, and the claim or decision it influenced. That helps a reviewer catch stale docs and irrelevant matches. Retrieval is a hint to verify against live files, not authority by itself.

Evaluating AGENTS.md, 2026 adds a second evidence boundary. A benchmark result that does not show every instruction file is harmful.

A procedural fix can be inspected, rehearsed, and improved. A warning without an owner or artifact usually disappears during the next busy sprint.

Prefer routed context over one permanent dump

Repository instruction research in 2026 found that instruction files increased prescribed exploration and testing cost by more than 20% in the reported benchmarks without improving success. That does not make instructions bad. It makes their content and routing testable engineering choices.

paqad-ai v1.67.0 was the current public release when this article was checked on July 21, 2026. Its repository describes risk routing, specialist roles, structured state, deterministic checks, documentation sync, and local audit records. These are product mechanisms, not independent outcome proof.

Use the AI workflow audit guide to map current practice, compare the bar with production-ready AI code, and use the AI coding consultant guide when the team needs outside ownership.

Sources have revisions
Inclusion reason recorded
Live file verified
Token budget visible
Failure types compared

Decision rule: Adopt a routed context package only when it improves task evidence or reduces cost without increasing the measured failure rate.

Frequently Asked Questions

How much context should a coding agent receive?

Enough to cover the task’s intent, affected modules, rules, interfaces, and proof methods, but no more by default. Determine the number experimentally for your repository rather than copying a universal token target.

Are AGENTS.md files harmful?

No universal conclusion follows from one benchmark. Instruction files can be valuable, but bloated or generic files may increase cost and exploration without improving task success. Test their actual contribution.

Should retrieval replace grep and file reads?

No. Retrieval can narrow candidates. The agent and reviewer should still verify important claims against current source files, especially when a decision affects security, data, or architecture.

What next?

Apply the context-package experiment scorecard to one current feature. If the repository cannot answer the questions it exposes, install paqad-ai and test a governed workflow with local evidence and explicit human authority.

Install paqad-ai from GitHub

Recognise this in your own team?

See how a change travels from request to live in one enforced process, then tell us about your team.