Using build failure conditions in TeamCity to ensure "improving" test coverage in .NET apps

Using build failure conditions in TeamCity to ensure "improving" test coverage in .NET apps

Monday 18 November 2013

All code that wasn’t written yesterday is legacy code. We all have it, and it’s important that we give it the love and attention it deserves. TeamCity (my favourite CI server, gun-against-head) lets you run code coverage metrics on your code-base using it’s own bundled install of dotCover - pretty common knowledge - but another thing it'll do is allow you to make your build fail if your code coverage decreases "build on build".

What this feature does, is track the coverage level of your "last successful build" vs. your current one, meaning you can make your build fail if you inadvertently reduce the level of coverage in your code-base. It's a nice way to introduce toxicity reports / tests to browner-field apps, as it doesn't mandate "THOU SHALT HAVE 90% COVERAGE" but instead, focuses on "improving" metrics over time. This kind of code sanitation is most valuable in brown-field apps where retrofitting testability hasn’t been the top priority – basically forcing build failures if you make crappy code crappier.

image

You can configure your project to do this by adding three build failure conditions, then add three metrics (percent of method coverage is less than 0.% compared to last successful build, percent of class coverage is less than 0.% compared to last successful build, percent of statement coverage is less than 0.% compared to last successful build) that “fail build on metric change” – the 0.5% threshold is good enough to cope with rounding errors or glitches, while maintaining that any new code that gets added, gets tested.

It’s a no brainer, and hopefully will stop you introducing regressions because something is “too hard” to test.