Last updated September 18, 2026
4 min read
A fictional pull request adds a package with a professional name, sensible API, and convincing README reference. The registry has no such release. The code review nearly approves a dependency that exists only in generated text. This is an explicitly fictional composite, not a customer result.
In this article
Package hallucination is a supply-chain concern
The USENIX Security 2025 study generated 576,000 Python and JavaScript samples across 16 models and found nonexistent package names. That scale shows a repeatable benchmark failure mode. It does not tell you the incident rate inside your production workflow.
USENIX Security 2025 package hallucination study supports the evidence boundary. 576,000 benchmark samples across 16 models are not a production incident rate.
| Item | Inspectable value |
|---|---|
| 1 | 576,000 samples |
| 2 | 16 models |
| 3 | 1 authoritative registry |
| 4 | 1 reviewed lockfile change |
Use the table as a diagnostic, not benchmark data. Replace illustrative entries with repository evidence before changing staffing, policy, or release authority.
Registry existence is only the first gate
Check the authoritative registry, resolved version, publication history, maintainer changes, package spelling, download pattern as a weak signal, license, known advisories, repository link, and lockfile delta. A real package can still be abandoned, malicious, or wrong for the task.
Sequence matters. When an early input is missing, later activity can look complete while a reviewer inherits the unresolved uncertainty.
Low-frequency APIs need targeted context
CloudAPIBench reported that targeted API documentation improved validity in its setting, while poor retrieval could reduce it. Give the agent the current package and internal API docs relevant to the task. Then verify calls against installed types or executable examples.
CloudAPIBench, ICSE 2025 provides a second check. A bounded cloud API benchmark with a particular model snapshot.
The fix is procedural because a procedure can be rehearsed and audited. A warning without ownership rarely survives a busy release week.
Attach a dependency receipt to review
A dependency receipt lists requested capability, candidates considered, selected package, exact version, registry evidence, lockfile change, provenance status, advisories, owner, and rollback. The reviewer can reject it without reverse-engineering how the name entered the diff.
paqad-ai v1.67.0 was the current public release when this article was verified on July 21, 2026. The repository describes local workflows, risk routing, structured state, checks, documentation sync, and audit records. Those are inspectable mechanisms, not independent proof of a business result.
Use the AI workflow audit guide for current-state mapping, compare the evidence bar with production-ready AI code, and read the AI coding consultant guide when outside support is relevant.
Decision rule: Reject any new dependency that cannot produce an authoritative registry match, exact lockfile resolution, and accountable owner.
Run the dependency-evidence receipt in a 45-minute working review with the product owner, module owner, and final reviewer. Do not turn it into a score. Read the decision rule aloud: Reject any new dependency that cannot produce an authoritative registry match, exact lockfile resolution, and accountable owner. Record disagreements, missing evidence, and the person responsible for the next action. Repeat the review after one real feature so the team can compare whether the artifact reduced ambiguity or only added paperwork.
Frequently Asked Questions
How common are AI package hallucinations?
Do not convert a benchmark rate into your production rate. The USENIX study establishes a repeatable failure mode across many generated samples. Measure your own proposed dependencies and block unverifiable names.
Is registry existence enough?
No. Attackers can publish typosquatted or dependency-confusion packages. Check spelling, provenance, maintainer changes, advisories, version history, and whether the package is actually necessary.
Should agents be allowed to add dependencies?
They can propose them. Require a deterministic receipt and qualified approval before a new dependency enters sensitive or production code. Existing approved packages may follow lighter policy.
What next?
Apply the dependency-evidence receipt to one active feature. If the repository cannot produce the required state and evidence, install paqad-ai and test the workflow locally with a named human decision owner.