If you’re about to make risky changes to software, with agents or without them, you want to start from a place of confidence. Here’s a small, slightly odd tool for buying some.
I frequently work with teams who are uncomfortable changing software they own. The reason is almost always the same: low test confidence, low test coverage, or both. Nobody wants to touch the thing that pays the bills when the only verification is “it worked last time we deployed it”.
This isn’t a new problem. It’s the problem at the heart of every legacy codebase, and the advice has been the same for twenty years: get it under test first, then change it. The trouble is that getting it under test is slow, boring, and expensive, so it rarely happens, and the codebase calcifies.
With agents accelerating the pace of change in software, we can’t risk making changes without a solid foundation of confidence in our tests.
A meta-harness for test uplift
I generated a tool to help me solve this problem - this is a agent-assisted test improvement harness - and while in it’s current version it’s tailored for .NET projects, the concept could trivially be applied more broadly to any language that supports code coverage tooling and optionally mutation testing.
TestImprovementAgent is a tool for .NET projects. It’s written in TypeScript (don’t lose your minds), and it acts as a meta-harness around your agent CLI of choice. Claude Code, OpenCode and the GitHub Copilot CLI are supported out of the box, and adding another is a single adapter file.
You point it at a repo, or a directory full of repos, give it a target, and walk away:
node src/cli.ts --src /abs/path/to/one-repo --target 90 --agent copilotThe harness instruments the codebase with Coverlet and Stryker.NET, measures where you are, and then delegates out to the agent to raise outside-in component test statement coverage up to your target. Once coverage is met, it moves on to mutation testing, handing surviving mutants back to the agent until the mutation score meets its own target.
The harness makes no model calls of its own. It’s a deterministic shell. It builds, runs the tests, measures coverage, decides what to ask for next, enforces the rules, and commits or reverts each iteration. If coverage goes up, the work is committed. If it goes down, it’s hard-reset. If the agent touches anything outside a test project, those changes are reverted and the agent is told why. The harness uses its own runsettings, so the agent can’t quietly weaken coverage exclusions to hit a number.
The agents only do the test uplift. They never get to judge whether the work is good. I’ve written before that anything that always has to be right shouldn’t be vibes - this is that exact technique - if the model writes code, deterministic tools decide whether it counts.
Mimicry, and only isolating what’s out of process
The harness ships with canned prompts that do their best to mimic your existing test style. If your tests look a particular way, the new ones should look like they belong next to them.
They also lean hard on one rule: only isolate out-of-process dependencies. Databases, HTTP calls, queues and the file system get stubbed. Everything inside the process boundary runs for real. Where there isn’t already a WebApplicationFactory or component test project, the agent scaffolds one, along with a short README mapping what it found.
There’s a strong focus on purely outside-in tests, rather than unit tests. Anything more granular would lock you into your current implementation. Hundreds of freshly generated unit tests that reflected exactly the shape of your current design would make your code harder to change, which is the exact opposite of what we’re trying to achieve. Outside-in tests care about what the system does, not how it does it.
Snapshots wearing component test pants
While these tests are stylistically “component tests”, they probably behave more like snapshot tests - they lock in what the program currently does. They don’t encode intent, and they don’t know whether the current behaviour is correct. This uses the big statistical dice roll machine for the thing it’s genuinely good at: fuzzy matching and replication. Look at this codebase, look at these tests, write more tests like these that exercise the bits nobody has exercised yet. That’s a task where “almost always right” is fine, because the deterministic shell will throw away changes that don’t fit.
Mutation testing deliberately stops this from being a “coverage-farming exercise”, because remember - Coverage is not correctness. Instead, a test suite that kills mutants is genuinely resilient to change in the code it covers.
YMMV
YMMV. This is paint so wet my hands are covered in it.
That said, I’ve done a few runs over some real-world apps, and it dragged coverage and mutation resilience up from around 60% to 98% in about an hour of spinning. Every iteration is a commit on a dedicated branch, with every prompt, transcript and tool output logged, so you can read exactly what it did and throw away anything you don’t like. If we’re now in a world where you want to make risky changes with agents, or do big traditional refactors by hand, starting from a place of confidence matters.
It’s been an interesting experience taking something I’ve always used primarily for human feedback, the tight loop of a TDD-style workflow, and using its artefact as a tool for automation instead. These aren’t “tests” as I previously used them. They weren’t formed intentionally to capture intent, but instead it’s using the same techniques to provide purely the safety net for change.
Given that, it’s still worth I’ll experimenting with any tool that gets a codebase, and it’s maintainers to strong foundations and a sense of confidence in making changes themselves.