In this article
Last updated September 18, 2026
4 min read
AI can generate implementation and tests in one polished pass.
That feels efficient because the artifacts agree. Agreement is useful only when both artifacts are anchored to the
right product behaviour.
If the code contains a mistaken interpretation, a later generated test can preserve the same interpretation and turn
it into a green check.
The answer is not a rigid ritual about writing test syntax first. The answer is to protect an independent source of
expected behaviour.
A new research warning about order
A July 2026 preprint titled “On the risk of coding before testing” studies whether erroneous generated code can
influence later test generation. The authors report that errors in the implementation can bias the generated tests and
propagate the original mistake.
This is new research. It needs replication, and its findings should not be treated as proof that every test generated
after code is unreliable.
It supports a sensible engineering principle: do not let the artifact being graded become the only answer key.
Create the evidence plan first
Before implementation, turn the specification into an evidence plan.
For each acceptance criterion, record:
- the observable starting condition;
- the action or event;
- the expected result;
- relevant edge cases;
- invariants that must remain true;
- the proof method;
- the person who owns any unresolved decision.
The evidence plan can be concise. Its purpose is not to predict every line of test code. Its purpose is to give the
checker a target independent of the implementation.
Split the roles
One practical workflow is:
- A requirements role clarifies behaviour and boundaries.
- A test-planning role derives proof methods from the approved specification.
- An implementation role receives the plan and scoped task.
- A test writer receives the specification and evidence plan, not the maker’s explanation as its primary source.
- A verifier maps results back to the approved criteria.
The roles can be people, agents, or a mixture. The important part is preserving independent inputs and incentives.
Test intent is broader than test code
Not every product promise belongs in an automated test.
A useful evidence plan may contain:
- a unit test for a calculation;
- a feature test for access control;
- a browser check for a visual interaction;
- a schema check for a contract;
- a registry lookup for a dependency;
- a performance measurement with a defined environment;
- a security review for an abuse path;
- a human decision for product behaviour.
Calling all of these “tests” can hide their different limits. Keep the evidence type visible.
Protect against implementation-shaped tests
Review generated tests for signs that they merely follow the code:
- Assertions repeat internal structure instead of observable behaviour.
- Fixtures contain only the happy path the implementation already supports.
- A test mocks the exact collaborator whose interaction needs proof.
- Boundary values come from the code’s constants instead of the specification.
- Error cases verify that an exception occurred but not that the correct user outcome followed.
- New behaviour appears in both code and tests without an acceptance criterion.
These patterns are not automatic defects. They are reasons to compare the test with the product promise.
Keep traceability visible
Assign stable identifiers to acceptance criteria and invariants. Link them to:
- the plan;
- the implementation;
- the tests or other evidence;
- review findings;
- documentation;
- the final receipt.
Now the workflow can report two valuable gaps:
Unproven promise: a criterion has no meaningful evidence.
Unrequested behaviour: changed code and tests have no approved criterion.
Reopen the specification when the target changes
Implementation often reveals a missing decision. Do not quietly update the tests to match the convenient answer.
Pause the work. Show the decision, realistic options, affected criteria, and evidence. Ask the named owner. Version the
specification and evidence plan before resuming.
This keeps change possible without letting the target move silently.
How paqad-ai supports the order
paqad-ai is designed to make planning, specification, proof methods, scoped development, review, and checks separate
workflow stages with durable artifacts. Higher-risk work can use a frozen specification and explicit human decisions.
The value is not that paqad-ai writes tests first by command.
The value is that the workflow preserves a target outside the generated implementation and keeps the proof chain
reviewable.
Generated code can be an excellent answer.
It should not be the source of the question and the answer key at the same time.
Install paqad-ai: https://github.com/Eliyce/paqad-ai/
Source notes
- On the risk of coding before testing, July 2026 preprint.
- Product mechanism: first-party details from the current
Eliyce/paqad-ai repository.