TL;DR
Leadership sees AI write a convincing demo and concludes that replacing a mature system is now cheap. Engineers need a way to decide and a definition of proof. Use AI to speed up a migration whose target is narrow and checkable, not as the reason to rewrite. Record what the old system actually does, replay real requests against both versions, compare the answers and move traffic slice by slice under a named owner. You end up with evidence instead of a demo.
In this article
- What the published AI migrations had in common
- Name the reason before choosing the method
- Define “the same” from the old system, not the new code
- Replay real traffic and compare the answers
- Switch traffic slice by slice, with a named owner who can stop it
- Where the AI Delivery Programme fits, and where it does not
- Frequently Asked Questions
8 min read
A rewrite is worth considering when you can name what it fixes and check, change by change, that customers see the same behaviour. AI can make the typing faster. It does not supply that check, and the published AI migrations that went well all had one in place before the model wrote a line.
The question reaches engineers in a recognisable form. In December 2025, the most senior engineer at a small company described on Reddit’s r/ExperiencedDevs forum a plan to have Claude rewrite a six-year-old Django application in Rust, prove identical output by having AI test every endpoint, and maintain the result with AI although nobody on the team reads Rust. A month later an engineer asked on Hacker News whether replacing a mature enterprise product with an LLM-built one is realistic, noting that early demos do not surface failure modes and that leadership was comparing a competitor’s ten-year end state with the first build cost of a new system.
What the published AI migrations had in common
The strongest public accounts are narrow. Google’s engineers reported in April 2025 on 39 migrations of identifiers from 32-bit to 64-bit integers, done by three developers over twelve months (Ziftci et al., arXiv, 2025). The model generated 74.45% of the 595 submitted changes, and the developers estimated that total migration time fell by half. Every generated file still had to compile and pass the existing regression tests or it was discarded, and every change was inspected by a developer and accepted by code owners.
Airbnb describes moving about 3,500 React test files from Enzyme to React Testing Library in six weeks against a manual estimate of a year and a half (Airbnb engineering, 2025). Each file had to pass its tests, lint and type checks before it counted, and the last few percent were finished by hand.
Both had a mechanical oracle: the build, the existing tests, a type checker. Translating a whole application between languages is harder. In an ICSE 2024 study, LLMs produced correct translations for between 2.1% and 47.3% of samples, and far fewer on real projects (Pan et al., ICSE 2024). A July 2026 paper on COBOL to Java validation goes the other way and treats the old program itself as the oracle, running both side by side and accepting a test only when their outputs match (Ferenczi et al., arXiv, 2026). That is the shape of proof a rewrite needs.
Name the reason before choosing the method
“AI makes it cheap” is a claim about build cost. The cost that sinks rewrites sits elsewhere: behaviour nobody wrote down. Joel Spolsky argued in April 2000 that old code carries years of bug fixes for cases you have forgotten, and that rewriting from scratch throws them away (Joel on Software, 2000). Thoughtworks’ Patterns of Legacy Displacement adds that workarounds harden into requirements and that behaviour hides in database triggers, batch jobs and integrations; teams that aim for full feature parity consistently underestimate it (Cartwright, Horn and Lewis, 2021).
So write the goal as something measurable. If it is latency, profile the slow paths first; a targeted change may meet it. If it is hiring or a dying runtime, the rewrite may be justified, but the team must be able to read, operate and debug the target language, because an AI cannot be paged at night. Then choose a method:
Comparison
| Approach | What it costs | Proof it allows |
|---|---|---|
| Incremental replacement behind the existing interface | Routing layer and some duplicate running; team learns the new stack slice by slice | Each slice compared and switched on its own |
| Parallel run with output comparison | Infrastructure to duplicate traffic and triage differences | Real inputs, both answers, every difference explained |
| Full rewrite and cutover | Lowest overhead until launch day, then all risk at once | Only what your tests already describe |
Define “the same” from the old system, not the new code
Equivalence has to be measured against what the old system actually does, including what looks wrong. Michael Feathers calls these characterisation tests: run the existing code, record its real output and keep that as the expectation until someone decides it is a defect (Feathers, 2016). For a web application that means recording responses, status codes, side effects on the database and messages sent, for a broad set of real inputs.
This is where “let AI test every endpoint” fails. If the same model writes the new code and the tests, both share its reading of the system, so the tests check the paths it understood and miss the ones it did not. If it derives tests from the old code, it can copy old bugs into the specification without anyone choosing to. Each of those can be fine; neither should happen unnoticed. The questions in The tests passed. Would they have caught the bug? apply directly, and so does the case for not letting the agent that wrote the change grade it.
Replay real traffic and compare the answers
Characterisation tests cover what you thought to record. Real traffic covers what customers actually send. Two published tools show the method. Twitter’s Diffy, open-sourced in September 2015, sends each request to the new version and to two copies of the old one; differences between the two old copies reveal noise such as timestamps, which is then ignored when judging the new version. GitHub’s Scientist, described in February 2016, ran old and new permission checks side by side in production, always returned the old answer, and logged every mismatch (GitHub, 2016). GitHub notes that some mismatches turned out to be bugs in the legacy code or data, and that the method suits reads only, not code with side effects.
Recorded traffic has a blind spot: it only contains what happened in the window you recorded. Month-end runs, annual renewals and rare error paths need their own cases.
Switch traffic slice by slice, with a named owner who can stop it
Martin Fowler’s strangler fig pattern, first described in 2004 and rewritten in August 2024, replaces a system piece by piece, with each small replacement carrying little risk and teaching you more about the old system (Fowler, 2024). Each slice can then follow the canary approach the Google SRE workbook describes: send a small share of traffic to the new version and compare its error and latency signals with the old one before widening (Google SRE workbook).
Agree in writing, before the first slice, what has to be true to move on:
- Characterisation tests for the slice pass against the new version, and they were reviewed by someone who did not write the new code.
- Replayed traffic shows no unexplained differences.
- Error and latency thresholds for the canary are set, with numbers.
- Rolling back to the old path has been tried, not just described.
- One named person has the authority to stop the migration, and the criteria for doing so are written down.
Where the AI Delivery Programme fits, and where it does not
The Programme is not a migration service and does not rewrite systems; companies that need software built or rebuilt are a poor fit for it. Its framework records that the agreed method ran on each change. It does not prove that two systems behave the same.
Where a team decides to modernise one application with AI coding tools, the Programme works on that application for five weeks. In the first, the team’s documentation, conventions and a delivery baseline are written down. In the second, the rules a script can test become deterministic checks. In the third and fourth, agreed backlog items go through that process with a record per change, so leadership sees what was checked instead of a sense of speed. No team has completed the Programme yet, and no before-and-after comparison exists. If that is the evidence you are missing, explore the Programme and ask for a fit conversation.
Frequently Asked Questions
Should we use AI to rewrite or migrate a legacy system?
Use it for the mechanical part of a migration you would undertake anyway. Before committing to the whole system, migrate one slice end to end and time all of it, including the differences you had to investigate and the cases finished by hand; that tail is the cost a demo never shows.
What is actually driving the rewrite, and could a narrower change meet that goal?
Ask for the goal as a number or a named risk: a response time, a runtime that loses support on a known date, a hiring problem with evidence. Then test the cheapest change that could meet it on the existing system. If it does, you have avoided a rewrite; if it does not, the measurement becomes the rewrite’s first acceptance criterion.
Which migration approach fits our risk, and what does each cost in team time and skills?
The skills cost is the one most often left out. Incremental replacement lets the team learn the new stack while the old one still serves customers. A full rewrite into a language nobody reads leaves the team unable to review the code it ships or to diagnose it in an incident, whoever generated it.
If AI writes both the new code and the equivalence tests, how could that verification fail?
It can test only the paths the model understood, assert whatever the new code happens to return, or copy old defects as requirements without a decision. A useful check on such a suite is to break the new code deliberately in small ways and see whether the tests notice; tests that survive every deliberate mistake are not evidence of equivalence.
What evidence should we require before switching traffic, and who owns the decision to stop?
Require written exit criteria per slice: reviewed characterisation tests, replayed traffic with every difference explained, canary thresholds with numbers and a rollback you have performed. The owner is a named person in your organisation with authority over the product, not the vendor and not the tool, and stopping must be an outcome they are allowed to choose.
