TL;DR
Tests can pass while the product is wrong when code and tests share the same mistaken interpretation of the requirement or omit an important promise entirely. Inspect observable product promise, requirement-to-test trace, independent examples, and test strength on one recent change. If another qualified reviewer cannot reach the same decision from the stored evidence, pause and repair the earliest missing handoff before increasing automation or output. The goal is dependable delivery that the team can explain, challenge, and maintain.
In this article
Last updated October 14, 2026
7 min read
Tests can pass while the product is wrong when code and tests share the same mistaken interpretation of the requirement or omit an important promise entirely. This question matters because faster generation moves work into decisions that still require evidence and ownership.
Passing tests are valuable.
They are also easy to ask to prove too much.
A green suite can show that the implementation satisfies the tests that ran in the environment recorded. It cannot, on its own, show that the tests represent the user’s need, cover every important edge case, or protect the correct invariant.
That gap becomes more important when the same AI system helps interpret the request, write the code, and generate the tests.
How code and tests can agree on the wrong answer
A product request says, “Members can invite colleagues.”
The implementation allows any signed-in member to create an invitation.
The generated tests confirm that a signed-in member receives a successful response and an invitation is stored.
Everything is green.
The product may still have intended only administrators to invite. The tests are consistent with the code because both share the same unapproved interpretation.
This is an illustrative example, not a claimed incident. It shows why internal consistency is weaker than independent evidence.
Begin with observable behaviour
Before implementation, write what a user or system should observe.
For example:
Promises to verify
- An administrator can invite a new member to an organisation.
- A non-administrator receives a forbidden response and no invitation is created.
- An expired invitation cannot be accepted.
- Removing the inviter does not silently transfer authority.
- Creating and accepting invitations produces the required audit events.
Each criterion should name a proof method:
- feature test;
- policy unit test;
- browser check;
- security review;
- audit-log assertion;
- human product decision.
The proof plan gives the test writer a source independent of the implementation.
Confirm invariants with a human
Some rules deserve stronger treatment because they must survive every code path.
Examples include:
- a user cannot approve their own elevated access;
- money cannot move without an immutable ledger entry;
- a deleted organisation cannot retain active invitations;
- personal data cannot appear in a public log.
An agent can help discover and phrase these rules. A named human should confirm them when they represent product, security, legal, or architectural intent.
Then carry stable identifiers for the invariant into tests, review, and the final evidence receipt.
Map promises to evidence in both directions
Forward traceability asks:
Which code and test prove each acceptance criterion?
Reverse traceability asks:
Which acceptance criterion or decision required each changed behaviour?
Forward mapping finds missing proof.
Reverse mapping finds unrequested additions.
This is important because an AI-generated test can make an extra behaviour look intentional simply by preserving it.
Test the tests
Mutation testing can help. A mutation tool introduces a small change, such as reversing a condition or changing a boundary, and runs the suite again. If the tests remain green, the mutation survived and the team has a reason to inspect whether the expected behaviour is protected.
Use mutation testing selectively. It has cost, and a surviving mutation is a signal rather than automatic proof of a defect.
The useful question is:
Would this test fail if the product promise were broken?
Keep evidence categories honest
A final report should distinguish:
- Test execution passed.
- Acceptance criterion has a linked test.
- Reviewer judges that the test meaningfully proves the criterion.
- Human confirms the product rule.
These are different claims.
The first can be deterministic. The third requires judgement. The fourth requires authority.
When a workflow labels them separately, a reviewer can see what is proven and what remains a decision.
What paqad-ai adds
paqad-ai is designed to connect specification obligations to implementation and test evidence. Its workflow can freeze the target for higher-risk work, keep acceptance criteria and invariants visible, run deterministic checks, use separate review roles, and report gaps before a human verdict.
No framework can make a weak product decision correct.
It can make the missing decision and missing evidence visible before a green suite becomes false confidence.
Tests are part of proof.
The product promise is the thing they are supposed to protect.
Trace one change through the bottleneck
Choose a recent change where observable product promise was disputed, late, or missing. Start from the accepted request rather than the final diff. Write down what the team knew before implementation, which boundary governed the work, and who could stop it. Then follow the change through planning, generation, verification, review, and release.
At each handoff, ask four concrete questions:
Run this in order
- Where is observable product promise recorded?
- Can a reviewer judge requirement-to-test trace without asking the original operator?
- Which artefact proves independent examples?
- Who owns test strength when the evidence conflicts?
Do not convert the exercise into a maturity score. The useful result is a short list of missing links. One may belong in repository guidance, another in CI, and another in the release decision. Repair the earliest missing link first because every later reviewer otherwise inherits the same ambiguity.
Run a second comparable change after the repair. If review becomes faster only because the same expert still explains the work orally, the process has not transferred. If another qualified person can reach the decision from the stored evidence, the change is becoming repeatable.
A usable review record for this decision should identify the accepted request, the exact revision being judged, the source of observable product promise, and the owner responsible for test strength. It should also preserve disagreement: a reviewer who finds weak requirement-to-test trace or missing independent examples needs a visible way to return the work as incomplete. That record gives the next developer a reliable starting point and prevents a confident summary from becoming the only explanation of what happened.
A promise-to-evidence map should include negative behaviour as well as the happy path. State what must never happen, such as exposing another user’s data, charging twice, or accepting an invalid transition, then identify the check that would reveal it. These negative promises often carry the greatest product risk and are easy to miss when tests are derived from the implementation. Giving them explicit owners helps reviewers distinguish a complete proof plan from a suite that merely exercises the code’s preferred route.
For a related view of the delivery system, read how AI-generated code changes review and how a reviewable AI feature plan works.
Sources used
- On the risk of coding before testing, July 2026 preprint that requires replication.
- OpenAI, Harness engineering, first-party account of mechanically enforced invariants and repository structure.
If your team is generating work faster than it can verify and own it, explore the AI Delivery Programme.
Frequently Asked Questions
How can tests pass while the product is still wrong?
Tests can pass while the product is wrong when code and tests share the same mistaken interpretation of the requirement or omit an important promise entirely. Judge the practice on one real change and keep the supporting evidence beside the work.
How do you test whether a test suite proves the requirement?
Track observable product promise, requirement-to-test trace, independent examples, and test strength together. A single speed or output number cannot show whether the team can safely accept and maintain the result.
Why can code and tests agree on the wrong behaviour?
Tests written from the implementation can repeat the same mistaken assumption as the code. Both may pass while the product promise, user need, or external contract remains untested.
Who should define the behaviour a test must prove?
The owner of the requirement should define observable behaviour and important invariants. Developers can choose test techniques, but they should not quietly redefine the product promise through implementation-shaped tests.
How do you test whether the test suite is meaningful?
Introduce controlled failures, inspect boundary cases, and trace each important requirement to independent evidence. If a plausible wrong implementation still passes, the suite is not proving enough.
What should happen when a requirement cannot be tested directly?
Record the limitation and agree on another proof method, such as contract checks, review evidence, monitoring, or a manual scenario. The gap should remain visible rather than disappearing behind green status.
