Building Supportable Systems (Deployment)

One of the biggest time-killers in software development is deployments (and environment management). I've worked on some big projects recently where people are spending many hours each week creating, maintaining and deploying software packages. These hours could be better spent fixing bugs, or adding value to business by adding features to the software. Instead, they're spent manually performing and tweaking installations of the software, often with additional overhead due to the inherent human errors along the way.

The other advantage to automatic deployment is that you can have high levels of confidence that you can quickly make changes to your software, build and deploy it to your test and production environments. The sooner you get that line of code into production, the sooner it's adding value to your business or customers. Recently I spent a whole weekend and re-factored one large web project with my colleague Sam (http://thwaitesy.com/) to make it simpler and quicker to package. We then set up an automated deployment process for the website. The deployment process developer 1-2 days a fortnight to do manually, now it takes about 10 minutes from code check-in, through automated build, and deployment to a developer-test environment. Instead of doing a deployment every 14 or so days, now we're getting well over 14 a day!

With the introduction of Microsoft Azure Websites and source control integration, I can't handle manually deploying websites any more, and for the vast majority of simple website projects this is a great way to go. For everything else... there's Octopus Deploy.

In my predominantly Microsoft-centric development career, nothing has been more of a game-changer than Octopus Deploy. It's a simple (and affordable) product that allows you to configure repeatable and configurable deployment processes for your software. Octopus Deploy is mainly for the deployment of windows server-based applications like windows services, websites, etc, though it has recently acquired support for Linux deployments via SSH (I assume OSX would work too). The heavy lifting is taken care of by an army of agents installed on your target machines and keeping with the cephalopod theme these are obviously "Tentacles" - though you can certainly have more than 8 of them.

The following diagram shows the main dashboard of Octopus Deploy where you can get a quick overview of your products (down the left hand side) and your environments (across the top). You can easily see which versions of each product are currently installed in each environment, if any have failed, or if any are in the process of being deployed.

Application packages are simply NuGet packages, optionally with some additional PowerShell scripts to help things along. Behind the scenes Octopus Deploy can store and use a variety of information about your applications by way of scoped variables. These can be substituted into the deployment process based on the environment, product, or even specific machine you are installing to.

The way this product works makes it really easy to work in an agile manner, quickly making changes to the deployment processes, or the variables and re-deploying quickly to an environment until the process is just right. Once you're happy with the process on your development or integration environment (for example) you can promote the deployment to the next environment (such as staging or production). The great benefit of this process is that you're not manually doing anything, and because it's repeatable, as you progress through your environments that you can have greater confidence of your production deployment being well tested. Of course the one caveat to this claim is that you need to keep your various environments reasonably similar in architecture to avoid unexpected changes at the end.

Corporate Features

One of the arguments I've had with the adoption of Octopus Deploy in a large enterprise was around how to control who can set up deployments, set variables, and subsequently deploy packages into various environments. Whilst Octopus Deploy has had an excellent role-based security system for a while, there was still the question of being able to enforce that a particular deployment progresses through the appropriate test and QA environments. In the recently released version of Octopus Deploy (2.6 / Dec 2014) they added a new "lifecycle" feature which addresses this very problem, and it's brilliant... mostly. The only downside of this new feature is that it works exactly as designed and I can't sneakily skip steps in the process like I did before, damn! :)

With these flexible security options and full auditing, it's really easy to give developers and testers access to the system so they can develop and test their own deployment processes without having to chuck the task over the fence to the operations team. At work we've given a few teams access to our Octopus Deploy instance and every time I look at the dashboard there are new applications being deployed to development and test environments. It's kinda great.

Extensibility and Integrations

I use Octopus Deploy in a couple of startups, open source projects and at work. Having used it across a variety of scenarios and scales I've found it to almost always work perfectly out of the box. In the few instances where the basic product hasn't had the ability to perform a deployment by default, I've always been able to achieve the goal through the addition of "Step Templates" (http://library.octopusdeploy.com), by adding a bit of custom PowerShell or through the fully-featured REST API.

The ultimate example of the integration process is to use TeamCity from JetBrains to perform automated build of your code, package it into a NuGet package and push it to Octopus Deploy for delivery. Once it's in Octopus Deploy you can perform automatically or manually triggered deployments and call PowerShell scripts to do things like publish to HipChat or Slack. Today I cloned the Slack notification script to make my own generic WebHook method (I'll publish this soon).

What's Next?

As part of a fun project at work I want to integrate a Netduino or Arduino with Octopus Deploy and this button (from Jaycar Electronics) so we can literally do "one button" deployments to production.

What could possibly be cooler (and geekier) than having this switch on the wall for the business-owner or CEO to push their new website to production (literally). I'll work out the technical details and put together a post on this ASAP. Keep calm, and deploy to production.

(Edit) This post has since been featured on CodeProject How We Stopped Wasting Time On Manual Deployments