The Anatomy of a Continuous Deployment System

The Anatomy of a Continuous Deployment System

Wednesday 13 February 2013

Continuous delivery is quite a popular discussion point at the moment. A lot of great people across multiple platforms are investing time and energy in it and those who aren’t often feel like they should be. Continuous delivery is an umbrella term for a set of practices that result in reducing the time to live of software features. It tends to be used to describe an overall approach of test automation, environment provisioning and automated software deployment. Software testing is a very important topic and there’s a tonne of great writing about adopting [T/B]DD out there, but what I really want to talk about is automated software delivery.

Your effectiveness as a software team can be measured as the time it takes for a feature request of an estimated size to be delivered from story to production. You always want to target the frequent delivery of high quality, reliable, software. The most effective way to do this is to ship small risk free packages, often.

Releasing software is a none trivial problem to a lot of people. It’s highly subjective. It’s highly environmental and it’s often risky. But it doesn’t have to be. In an ideal world, the act of deploying any piece of your software to any target environment should always be repeatable, reliable and quick. It’s essential to attain these qualities to ensure that the feedback loop between your programmers and your users is tight, effective and reactive.

For the scope of this discussion, I’m not going to discuss test driven development or continuous integration, and instead presume they exist already. It’s worth signposting that in order for a continuous deployment system to be the most effective it can be, it should be paired with automatic environment provisioning.

A continuous deployment system conceptually consists of four main pillars:

  • Continuous integration and build
  • Software packaging
  • Software transporting
  • Software installation
The responsibly of these four main components are distinct and independent.
  • A continuous integration server retrieves recent commits from source control and builds a software artifact.
  • A software packaging script can then operate on the output of a continuous build and produce a versioned, installable package.
  • Some transportation mechanism should then be responsible for moving this package from a build server and distributing it to application servers.
  • Then an installation script or application should be responsible for extracting and installing this package.
The four pieces of a continuous deployment system may be a single application suite, or it may be a loose association of smaller applications and scripts, however conceptually, any one of those four pillars should be exchangeable for another piece of software that does roughly the same job in a compatible way.

There are some practices worth considering when building a system for this purpose:

It’s very important that your continuous build outputs your application along with installation scripts that can be executed silently, regardless of technology or platform choice.

You should ensure that all of your packages are produced “configuration neutral”, with enough automation inside the package to configure them for the environment they end up on when they arrive. It’s worth making sure any configuration files are tokenised and scripts are provided to finalise setup on the target environment.

You must always ensure that your package installs are idempotent. Multiple installations should result in the same outcome.

When putting a high level of automation in place, it’s advisable to have some kind of radiator of information that gives you an overview of what software is where. As your deployment frequency increases, this information will become increasingly important.

You will probably end up bending and changing your applications to accommodate deployment. You’ll want to keep your applications as environment and platform agnostic as you can to achieve this. No hardcoded paths outside of your applications root or presumptions about environmental settings.

All delivery ecosystems, regardless of platform or technology choice will have these concerns. Some frameworks or platforms make these more trivial problems to solve than others, but if you’re looking to get into continuous delivery and deployment. Some modern cloud platforms have these concepts baked in from the start (e.g. Heroku, AppHarbor and WindowsAzure).

Regardless of your choice, you will have to address each of these themes.