Last updated September 18, 2026
4 min read
In this article
AI development reports often make every successful result look equal.
Tests passed. Architecture passed. Security passed. Specification passed. Ready to merge.
The uniform green presentation hides different kinds of knowledge.
A test runner can prove that a command exited successfully in a recorded environment.
A model can judge that an architecture looks coherent.
A human can decide that the available evidence is sufficient for production.
These claims support one another. They are not interchangeable.
Four useful evidence categories
Deterministic evidence
The same defined input and method should produce a checkable result.
Examples:
- lint, type, test, and build commands;
- schema validation;
- file existence;
- package registry query;
- dependency-direction rule;
- hash comparison;
- static pattern check.
Record the command or method, environment, time, exit code, and output reference.
Deterministic does not mean complete. A passing suite proves only what the suite and environment cover.
Measured evidence
Measurements depend on method and conditions.
Examples:
- response latency;
- memory use;
- coverage;
- mutation score;
- complexity;
- accessibility or performance audit.
Record the instrument, environment, sample, baseline, and limitation. A number without method is decoration.
Model-judged evidence
Some questions require interpretation:
- Does the plan fit the architecture?
- Does a test meaningfully cover an acceptance criterion?
- Is the design consistent?
- Which threat is likely?
- Does documentation explain the changed behaviour?
A model can perform useful specialist analysis. Record its inputs, reasoning, uncertainty, and claim boundary. Do not present the conclusion as a deterministic pass.
Human-decided evidence
Some outcomes are choices rather than discovered facts:
- which product behaviour should apply;
- whether to accept a risk;
- whether an architecture exception is justified;
- whether the available evidence supports production.
Record the authorised owner, alternatives, evidence considered, reason, and time.
Add a fifth state: unknown
Mature systems can say “inconclusive.”
Use it when:
- evidence is missing;
- sources conflict;
- a check did not run;
- the evaluator lacks capability;
- the decision owner is unavailable;
- the claim extends beyond the measurement.
Unknown should route to more evidence, a human decision, re-planning, or a stop. It should not be converted to green by a summary.
Build reports around claims
For every material conclusion, record:
| Field | Question |
|---|---|
| Claim | What exactly is being asserted? |
| Category | Deterministic, measured, model-judged, human-decided, or unknown? |
| Source | Which artifact, command, or person supports it? |
| Scope | Which files, environment, behaviour, or period does it cover? |
| Limit | What does it not establish? |
| Status | Pass, fail, inconclusive, or decision required? |
This makes the report inspectable rather than merely reassuring.
Examples of honest boundaries
“All tests passed in CI” does not prove every acceptance criterion has a meaningful test.
“The registry contains the package” does not prove the package is legitimate or suitable.
“No high-severity issue was found by the model reviewer” does not prove the code is secure.
“The reviewed hash matches” proves artifact identity, not content correctness.
“The human approved” does not prove meaningful oversight unless the person had evidence and authority.
How paqad-ai uses evidence
paqad-ai is designed to run deterministic checks, store structured artifacts, coordinate specialist reviews, pause for human decisions, and create an evidence receipt.
Its strongest position is not that every result becomes certain.
It is that the workflow can say how each result was produced and where its proof ends.
Trustworthy AI development does not require pretending uncertainty disappeared.
It requires making uncertainty and evidence legible enough to decide responsibly.
Explore the paqad-ai framework on GitHub ↗
Source notes
- NIST AI RMF Playbook, Measure, voluntary measurement and oversight guidance.
- Product mechanism: first-party details from the current Eliyce/paqad-ai repository.