TL;DR
Route models by the consequence and evidence needs of the task, not by a permanent cheap-versus-smart label. Start with a small task set, compare accepted output, correction effort, latency and total cost, and keep a fallback for uncertain results. A cheaper call saves money only when it does not create more review, retries or escaped defects. This keeps routing tied to evidence rather than reputation.
In this article
6 min read
These two reader concerns frame the article: How should you route software-development tasks between AI models? and Is it worth using cheaper models for simpler coding-agent steps?. The links show the wording and context of the questions; the answer below uses primary or official sources for material claims.
Weak signal and stronger evidence
| Weak signal | Stronger evidence |
|---|---|
| The agent or tool says the work is complete | The named result is checked against the exact revision |
| A policy exists in a document | The rule is visible in the working delivery path |
| More output is produced | Accepted outcomes improve without hidden review debt |
Task classes matter more than one model leaderboard
Separate classification, retrieval, code drafting, transformation, test ideas, review, architecture, and final decisions. A small local model may handle labels well. A stronger model may justify cost on complex planning. A deterministic script may be the correct choice when no model judgment is needed.
OpenAI on harness engineering supports the boundary. A vendor internal case study, not a universal productivity result.
| Evidence point | Inspectable value |
|---|---|
| 1 | Four decision lenses |
| 2 | Seven measurable signals |
| 3 | Three fallback classes |
| 4 | One retained route receipt |
Use this as a diagnostic. Replace illustrative entries with current repository evidence before making a policy, staffing, or release decision.
Test routing with the same tasks and acceptance bar
Build a small evaluation set from work the team actually performs: a classification, a documentation edit, a bounded refactor, a failing-test diagnosis and a change that needs architectural judgment. Remove sensitive data and preserve the expected result, risk lane and review method. Run every eligible route against the same cases.
Record the model and version, context supplied, tools allowed, output, check results, correction turns, reviewer minutes, latency and total call cost. Judge accepted results rather than stylistic preference. One route may be cheap for classification and expensive after correction on refactoring; another may justify its price only when the failure consequence is high.
A route is a policy decision
| Routing question | Evidence required |
|---|---|
| Is the task eligible? | Data boundary, tool need and consequence class |
| Did the route succeed? | Acceptance checks on the exact output |
| What did success cost? | Calls, retries and human review time |
| Can failure be detected? | Validated malformed, uncertain and wrong-result cases |
| What happens next? | Safe fallback that preserves the risk floor |
Do not use a single blended score. A route that is cheaper but violates the privacy boundary is ineligible. A route that is fast but produces failures the evaluator cannot detect is unsafe to automate. Keep each floor visible, then optimise among the eligible choices.
Evaluate four outcomes together
Measure task success, correction turns, reviewer time, tokens or price, latency, privacy boundary, and failure severity. A cheap route that doubles expert review is not cheap. A high-quality route that violates data policy is not eligible.
- Define task classes. Record the owner, artifact, and completion evidence before the next transition.
- Benchmark eligible routes. Record the owner, artifact, and completion evidence before the next transition.
- Set risk floors and fallback. Record the owner, artifact, and completion evidence before the next transition.
- Review results by version. Record the owner, artifact, and completion evidence before the next transition.
Sequence matters because later evidence inherits the intent and scope established earlier. A skipped step transfers uncertainty to the final reviewer.
Fallbacks need explicit risk behavior
Define what happens on timeout, low confidence, malformed output, provider outage, or missing capability. Fallback can move to another model, a deterministic tool, or a human queue. It must not silently downgrade a sensitive task to weaker evidence.
Anthropic on context engineering provides a second evidence boundary. Vendor engineering guidance, not comparative outcome research.
Mistake: Choosing one flagship model for every task. Fix: Route by measured task outcome and policy, with deterministic tools and humans treated as valid destinations.
The fix is procedural so the team can rehearse and audit it. A warning without ownership disappears when delivery pressure returns.
Routing policy should improve from retained evidence
OpenAI’s harness engineering case study shows how constraints and feedback systems can support high agent output in its internal setting. Anthropic’s context guidance emphasizes signal selection. Neither establishes a universal routing table. Use local evaluated results and keep model versions in the receipt.
Use the AI workflow audit guide for current-state mapping, compare the bar with production-ready AI code, and read the AI coding consultant guide when outside ownership is relevant.
- Task class explicit
- Model version pinned
- Cost and review counted
- Privacy floor met
- Fallback cannot weaken risk policy
Decision rule: Change a route only when repository evidence improves quality, cost, latency, or risk without weakening another required floor.
Run the task-to-model routing scorecard in a 45-minute working review with the product owner, module owner, and final reviewer. Do not turn it into a score. Read the decision rule aloud: Change a route only when repository evidence improves quality, cost, latency, or risk without weakening another required floor. Record disagreements, missing evidence, and the person responsible for the next action. Repeat the review after one real feature so the team can compare whether the artifact reduced ambiguity or only added paperwork.
Review routing failures by cause
When a route fails, separate model capability from missing context, a weak evaluator, tool failure and an unsuitable task definition. Moving every failed case to the largest model hides process defects and makes the policy expensive without making it informative. Re-run one variable at a time: the same model with better context, the same context with a different model, or the same output with a stronger check.
Keep an exception path for urgent work, but give it an owner and expiry. A temporary manual route should not become permanent policy because its result was never compared. Publish the current routing table with the evidence date and eligible task classes so developers know what is tested and what remains experimental.
The final receipt should explain why the selected destination was eligible, which alternative was considered, what evidence supported the choice and what triggered fallback. That turns routing from an invisible cost optimisation into a reviewable delivery decision.
Before rollout, let developers see the route and challenge it. A hidden router makes correction patterns hard to explain. A visible reason code such as privacy boundary, complex planning or deterministic transformation helps reviewers spot misclassification and supplies cleaner evidence for the next policy revision.
Decision rule
Proceed only when the team can name the owner, inspect current evidence and stop or reverse the work without reconstructing an agent conversation.
Paqad is a first-party Eliyce product. Teams that want to test these controls on their own repository can explore the AI Delivery Programme.
Frequently Asked Questions
How should you route software-development tasks between AI models?
Classify the task by consequence, data boundary, tool needs and proof method. Compare eligible models on accepted output, correction work, reviewer time, latency and total cost. Keep the route versioned and send uncertain or ineligible work to a safer model, deterministic tool or human owner.
Is it worth using cheaper models for simpler coding-agent steps?
Yes when the task is narrow, failure is detectable and local evaluation shows that savings survive retries and review. A lower call price is misleading when experts must repair the result or inspect it more carefully.
How large should the routing evaluation set be?
Start with enough real cases to cover each intended task class and its important failure modes. Add cases when reviewers find a new error pattern. The set should expose route differences and regressions; it does not need to imitate every repository task.
What should happen when the preferred model is unavailable?
Use a pre-approved fallback for that task class. The fallback may be another model, a deterministic process or a human queue. It must preserve privacy, permission and evidence requirements and should never silently downgrade a sensitive task.
When should a routing decision be sent to a human?
Escalate when the task falls outside the evaluated class, the result is inconclusive, sensitive data changes eligibility, models disagree on a consequential finding or the fallback would weaken a required control.
How often should the routing policy be retested?
Retest when a model version, prompt, tool, context source, price or evaluator changes, and after repeated corrections or an incident. Preserve the old results so the team can distinguish a real improvement from a changed test.
