TL;DR
When someone calls AI-generated code production-ready, ask what production condition has been demonstrated. Passing tests are only one part of the answer. Require evidence for the product behaviour, data and permission boundaries, deployment path, monitoring, rollback and ownership after release. Replace the vague label with a release decision: what was checked on the final commit, what remains uncertain, who accepts that risk and what signal will trigger a rollback.
In this article
- Turn “production-ready” into six answerable questions
- Separate code confidence from release confidence
- Ask for evidence from the exact release candidate
- Test the production boundary, not a convenient substitute
- Define the stop signal before release
- Keep responsibility with the team that releases it
- Frequently Asked Questions
7 min read
“It works” and “it is ready for production” are different claims. AI can produce code, tests and a confident summary from the same interpretation of a task. Those artifacts may agree with one another while missing the real user need or the conditions found in production.
So how do you challenge a claim that AI-generated code is production-ready? Do not argue about whether AI code is good or bad. Ask for the operational evidence that would be required from any change: correct behaviour, protected boundaries, deployability, observability, reversibility and a named owner.
Turn “production-ready” into six answerable questions
The phrase causes trouble because people use it as a conclusion without naming its criteria. Replace it with questions that can be checked.
- Behaviour: does the change satisfy the acceptance examples that matter?
- Boundaries: are permissions, validation, data rules and failure paths protected?
- Deployment: can the exact artifact move through the real release path?
- Observation: will the team know when the change is failing or harming users?
- Recovery: can the change be disabled, rolled back or repaired safely?
- Ownership: who responds when production behaves differently from the test environment?
A “yes” needs evidence. “The agent ran the tests” is not evidence for monitoring, data migration or rollback. “The code looks clean” does not demonstrate an authorisation boundary. The strength of the evidence should follow the cost and reversibility of failure.
This reframing also makes disagreement productive. The author and reviewer can discuss a missing condition rather than debating a broad label.
Separate code confidence from release confidence
Code review asks whether the implementation is understandable, consistent with the design and free of visible defects. Release review asks whether the change can enter a particular environment safely. One informs the other, but they are not interchangeable.
Consider a schema change. Its code may be small and well tested. Production readiness still depends on the size and shape of existing data, lock behaviour, compatibility with the previous application version and a recovery route if the migration stops halfway. Those questions cannot be answered by reading the new model class.
Comparison
| Claim | Evidence that helps | Missing question |
|---|---|---|
| “Unit tests pass” | Logic behaves under represented inputs | Were integration and failure paths exercised? |
| “The agent reviewed it” | A second automated pass found no reported issue | Who checked product intent and domain risk? |
| “Staging worked” | One environment accepted the release | Does staging represent production data and traffic? |
| “We can roll back” | A mechanism exists | Was recovery rehearsed with this change? |
GitHub’s documentation on Copilot code review says it may miss problems and its feedback should be validated carefully. Its reviews are comments by default rather than approvals that satisfy required human review. The product boundary reinforces a sound team boundary: automation can produce findings; accountable people make the release decision.
Ask for evidence from the exact release candidate
Evidence becomes stale when another commit, dependency resolution or build step changes the artifact after testing. Record the commit and artifact that passed each required check. If a fix lands after review, rerun the affected checks and make the new state visible.
GitHub required status checks operate on a specific commit. The same discipline should apply to manual checks and demonstrations. A screenshot without the version, a test result copied from another branch or a verbal “we checked that” cannot establish the state of the release candidate.
For a meaningful evidence packet, record:
- the exact change identifier;
- the behaviours and risks each check addresses;
- the command, environment or procedure used;
- the result, including an inspectable artifact where useful;
- the remaining uncertainty and its owner.
Evidence should be small enough to review. Do not bury the decision in a transcript or attach hundreds of lines of logs without explaining which signal matters.
Test the production boundary, not a convenient substitute
AI-generated changes often arrive with generated unit tests because those are easy to create locally. Unit tests are useful, but a release fails at boundaries: authentication, external services, queues, data, configuration, timeouts, concurrency and deployment order.
Map the change to the boundaries it crosses. A new endpoint needs permission and validation checks. A background job needs retry, idempotency and dead-letter behaviour. A data change needs compatibility and recovery. A third-party integration needs timeout and degraded-mode decisions.
Then choose the smallest realistic check for each important boundary. You may use an integration test, a production-like rehearsal, a contract test, a migration dry run or a controlled release. The technique depends on the risk. The requirement is that the evidence address the real failure rather than a simplified proxy.
Decision guide
- Reversible, isolated change: focused automated checks plus inspection may be enough.
- Shared contract or persistent data: add compatibility, migration and recovery evidence.
- Permission, payment or destructive action: require explicit negative tests and domain review.
- High uncertainty in production shape: use a limited release, strong observation and a defined stop signal.
Define the stop signal before release
Monitoring is useful only when the team knows what it will do with the signal. Name the expected healthy behaviour, the failure signal, the person watching it and the action that follows. “We will monitor it” leaves every decision until the release is already under pressure.
A stop signal may be an elevated error rate, a queue that stops draining, an authorisation anomaly, a failed reconciliation or a product result outside an agreed bound. It should connect to the change rather than rely only on a general infrastructure dashboard.
Recovery also needs a truthful path. A rollback may be unsafe after data has been transformed or external effects have occurred. In that case, the team may need a feature flag, a forward repair, a compatibility period or a manual reconciliation plan. Calling any of those “rollback” without checking the real sequence creates false confidence.
Keep responsibility with the team that releases it
The agent does not carry a pager, explain a business rule to support or restore damaged data. The team accepting the change needs a person who understands the release decision and can coordinate the response. That person does not need to have typed every line, but they must be able to explain the behaviour, evidence and recovery choice.
This is also why a polished agent summary is insufficient. Summaries help navigation, but ownership comes from inspecting the artifacts and making the judgment. The AI code evidence receipt explains how to retain that decision after the session ends, and the article on reviewing AI-generated code covers the earlier code-review pass.
The strongest pushback is calm and specific: “Show me the condition, the evidence and the owner.” If one is missing, describe the work needed before release. You do not need to prove the code will fail. The person claiming readiness needs to show why the team should accept the risk.
If your engineering team needs a repeatable production-readiness gate for AI-assisted work, Fix this for your team. The Programme applies a shared workflow to real backlog changes and compares it with your starting point; it does not guarantee a particular delivery result.
Frequently Asked Questions
How do you challenge a claim that AI-generated code is production-ready?
Ask which production conditions were demonstrated: correct behaviour, protected boundaries, deployability, observation, recovery and ownership. Require evidence tied to the exact release candidate, then record the uncertainty that remains and the person accepting it.
What does production-ready actually mean for AI-generated code?
It means the team has enough evidence to release a particular change into a particular environment with an understood risk. It is contextual. A reversible internal tool change and a payment migration require different checks, reviewers and recovery plans.
Are passing automated tests enough to approve a production release?
No. Tests support the decision, but their value depends on what they exercise and assert. Release confidence may also require permission checks, real integration boundaries, production-like data, deployment rehearsal, monitoring and a workable recovery route.
Who owns a production problem caused by AI-generated code?
The team and named humans who accepted and released the change own the response. The agent is a tool used during development. Responsibility cannot be assigned to a model that cannot assess business consequences or operate the service after release.
What should a reviewer ask when staging looks fine?
Ask how staging differs from production in data volume, configuration, traffic, permissions and integrations. Then identify which differences matter for this change. A successful staging run is useful evidence only for the conditions staging represents.
When should a team delay an AI-assisted release?
Delay when an important boundary has no evidence, the tested artifact differs from the release candidate, the remaining risk has no owner, or recovery is only an assumption. State the missing condition clearly so the delay leads to a specific piece of work.
