TL;DR
Run the readiness audit on one real repository and one bounded change before giving an AI coding tool wider access. Test whether the team can state the requirement, load current rules, limit permissions, verify the result, recover from failure and preserve a reviewable record. A questionnaire shows what people believe is ready. An executable audit shows which controls still work when the tool changes code.
In this article
7 min read
Two reader questions guide this article: How do you run an executable readiness audit before introducing AI development? and How do we know if our codebase is ready for AI coding tools?. The first reflects repository-readiness language used by practitioners. The second is the practical decision behind NIST’s broader guidance on governing, measuring and managing AI risk.
An executable audit follows one change through the actual delivery path. It does not ask whether a policy exists. It asks whether the right person, instruction, permission, check and recovery action appear when that change needs them.
Questionnaire and executable evidence
| A questionnaire can record | An executable audit must demonstrate |
|---|---|
| The team says requirements are documented | The agent receives the current requirement and unresolved decisions are visible |
| Access should follow least privilege | The audit run cannot reach an unapproved system or secret |
| Code should be tested | Named checks run against the exact changed revision |
| A human remains accountable | A named person can approve, reject or stop the run before commitment |
| Failures should be recoverable | The team can restore a bounded state and resume from a durable record |
Choose one repository and one representative change
Start with a repository that matters enough to expose real constraints but can be tested without risking a customer or production system. Avoid a toy repository with perfect documentation. It will hide the missing ownership, stale commands and unclear boundaries the audit is meant to find.
Choose a change that crosses several ordinary handoffs. An illustrative example is adding a lockout notice to an account-recovery flow. The change is small, but it still needs approved wording, the owning module, an authentication rule, tests, documentation and a release decision. The audit does not need to deploy the change. It needs to show whether the team can carry it from request to review without filling gaps through guesswork.
Record the starting revision and environment before the tool acts. If those identities are missing, later evidence cannot prove what was inspected or tested.
Run the audit as five evidence-producing passes
Executable readiness audit
- Frame the change. Write the user-visible outcome, acceptance conditions, exclusions and the person who can settle ambiguity. Ask the tool to restate the task and flag unresolved decisions before editing.
- Load repository context. Provide the current architecture boundary, local instructions, nearest working pattern, commands and documentation owner. Check which sources the tool actually used instead of assuming the largest context is the best context.
- Exercise permissions. Use a disposable branch or isolated environment. Allow only the files, commands and services needed for this change, then attempt one forbidden action to confirm the boundary is real.
- Verify the result. Run deterministic checks against the exact revision, review the changed behavior and inspect whether unrelated files or rules moved. Separate a failed check from an absent check.
- Test interruption and handoff. Stop the run before completion. Confirm that another person can find the approved task, current state, results, unresolved question and permitted next action without reading the full agent conversation.
Each pass produces evidence or a named gap. A confident explanation from the tool is not evidence that the repository, permissions or result are ready.
Separate deterministic checks from decisions
Scripts can confirm that required files exist, commands exit successfully, schemas validate, dependencies are pinned and results belong to the current revision. They cannot decide whether account-lockout wording is acceptable, whether an authentication trade-off is justified or whether the remaining risk is worth releasing.
Assign each audit item an evidence type and an owner. This prevents a model-written assessment from being treated as a policy decision.
| Audit item | Evidence to retain | Accountable role |
|---|---|---|
| Requirement | Approved outcome, exclusions and unresolved questions | Product owner |
| Repository context | Source paths, rule versions and working commands | Module owner |
| Access | Granted scope and result of a denied-action test | Platform or security owner |
| Verification | Exact revision, check commands, results and reviewer findings | Technical reviewer |
| Recovery | Restore point, preserved state and tested resume action | Platform owner |
| Release | Decision, accepted gaps and expiry condition | Named release authority |
NIST’s AI Risk Management Framework Core describes Govern, Map, Measure and Manage as connected functions. It calls for clear roles, documented testing, uncertainty and traceable management decisions. NIST also says its actions are not a checklist or ordered certification path. The framework supports this audit’s emphasis on evidence and ownership; it does not certify a repository or an Eliyce product.
Test failure paths while the stakes are low
A clean run proves less than teams often assume. Add one controlled failure to each important boundary:
- remove a required piece of product context and check whether the tool asks rather than invents;
- request a file outside the approved scope and confirm access is denied;
- make a test stale or point it at the wrong revision and confirm freshness is checked;
- interrupt the run and ask another reviewer to resume from stored evidence;
- create a result that passes automation but violates an acceptance condition and confirm human review catches it.
Do this in an isolated branch or test environment. The purpose is to verify the control, not to create a production incident.
End with a bounded readiness verdict
Use three verdicts:
- Ready for the tested task class: every required floor produced current evidence, named owners accepted the remaining limits and the same boundaries will apply to the next run.
- Needs attention: the missing controls and their owners are known, and work remains paused until the evidence exists.
- Inconclusive: the audit could not exercise a meaningful boundary or the result depends on an unresolved decision.
Tie the verdict to the repository, revision, tool configuration, permission set and task class. “Ready” for documentation-only edits does not imply readiness for authentication changes. Set an expiry trigger such as a provider change, repository reorganisation, new production access or a serious failure.
Keep the result with the engineering evidence rather than in an audit slide deck. Eliyce’s delivery record shows how requirements, checks, decisions and remaining gaps can stay attached to one change. Teams comparing process controls with assistant features can also use enforced process versus tools alone.
Where Paqad fits
Paqad is a first-party Eliyce product for structuring requirements, roles, checks, documentation and decisions around software changes. Those mechanisms can host parts of an executable audit. They do not decide product intent, prove that a team’s tests are sufficient or turn NIST guidance into certification.
Decision rule
Introduce wider AI development only after one representative change proves that current rules load, permissions hold, results can be checked, a human can stop the work and another person can recover from the recorded state.
Teams that want to test this operating model on their own repository can explore the AI Delivery Programme.
Frequently Asked Questions
How do you run an executable readiness audit before introducing AI development?
Select one representative change, record its starting revision and run it through requirement, context, permission, verification and handoff checks. Treat every missing artifact or unclear owner as a finding. The audit passes only for the task class and access level that were actually exercised.
How do we know if our codebase is ready for AI coding tools?
Readiness exists when a tool can find current intent, work inside a bounded scope, run trustworthy checks and leave evidence another person can inspect. A good repository also exposes uncertainty early: the agent can identify who must decide instead of silently completing the requirement itself.
Which repository should we audit first?
Choose a maintained repository with real delivery rules, active owners and a change you can isolate safely. A neglected system may be too risky for the first audit, while a toy repository hides normal friction. The best starting point has meaningful constraints and a reversible test path.
How can we test permission boundaries without putting production at risk?
Clone the relevant shape of the environment with synthetic data and deliberately request one prohibited file, secret or command. Record the denial at the tool and infrastructure layers. A written instruction to avoid production is weaker than an environment that makes the action impossible.
What should block rollout even when most readiness checks pass?
Block rollout when any required safety floor is missing: no accountable owner, unresolved sensitive behavior, unbounded credentials, checks that cannot identify the tested revision, or no recovery path. These gaps cannot be offset by strong documentation or a high aggregate score.
When should we run the readiness audit again?
Repeat the affected passes when the repository structure, task class, model provider, tool permissions, verification path or accountable owner changes. Also rerun after an incident or repeated reviewer correction. A calendar reminder helps, but a material change is the stronger trigger.
