Comparing Application Deployment: 2005 vs. 2015

Note: Check out the Latvian Translation.

Over the past 10 years the ways we build and deliver applications has changed significantly. It seems like much of this change has happened overnight but don’t worry, it is perfectly normal to look up and feel disoriented in the 2015 deployment landscape.

This article compares the deployment in 2005 with “modern” deployment so that all the new terms and techniques will make sense. Forewarning: My background is primarily Java / JVM so I will use that terminology but try to make the ideas polyglot.

2005 = Multi-App Containers / App Servers / Monolithic Apps
2015 = Microservices / Docker Containers / Containerless Apps

Back in 2005 many of us worked on projects that resulted in a WAR file – a zip file containing a Java web application and its library dependencies. That web application would be deployed alongside other web applications into a single app server sometimes called a “container” because it contained and ran one or more applications. The app server provided a bunch of common services to the web apps like an HTTP server, a service directory, and shared libraries. Unfortunately deploying multiple apps in a single container created high friction for scaling, deployment, and resource usage. App servers were supposed to isolate an app from its underlying system dependencies in order to avoid “it works on my machine” problems but things often didn’t work that smoothly due to differing system dependencies and configuration that lived outside of the app server / container.

In 2015 apps are being deployed as self-contained units, meaning the app includes everything it needs to run on top of a standard set of system dependencies. The granularity of the self-contained unit differs depending on the deployment paradigm. In the Java / JVM world a “containerless” app is a zip file that includes everything the app needs on top of the JVM. Most modern JVM frameworks have switched to this containerless approach including Play Framework, Dropwizard, and Spring Boot. A few years ago I wrote in more detail about how app servers are fading away in the move from monolithic middleware to microservices and cloud services.

For a more complete and portable self-contained unit, system-level container technologies like Docker and LXC bundle the app with its system dependencies. Instead of deploying a bunch of apps into a single container, a single app is added to a Docker image and deployed on one or more servers. On Heroku a “Slug” file is similar to a Docker image.

Microservices play a role in this new landscape because deployment across microservices is independent, whereas with traditional app servers individual app deployment often involved restarting the whole server. This was one reason for the snail’s pace of deployment in enterprises – deployments were incredibly risky and had to be coordinated months in advance across numerous teams. Hot deployment was a promise that was never realized for production apps. Microservices enable individual teams to deploy at will and as often as they want. Microservices require the ability to quickly provision, deploy, and scale services which may have only a single responsibility. These requirements fit well with the infrastructure provided by containerless apps running on Docker(ish) Containers.

2005 = Manual Deployment
2015 = Continuous Delivery / Continuous Deployment

The app servers of 2005 that ran multiple monolithic apps combined with manual load balancer configurations made application upgrades risky and painful so deployments were usually done sparingly in designated maintenance windows. Back then it was pretty much unheard of to have a deployment pipeline that fully automated delivery from an SCM to production.

Today Continuous Delivery and Continuous Deployment enable developers to get code to staging and production sometimes as often as tens or even hundreds of times a day. Scalable deployment pipelines range from the simple “git push heroku master” to a more risk averse pipeline that includes pull requests, Continuous Integration, staging auto-deployment, manual promotion to production, and possibly Canary Releases & Feature Flags. These pipelines enable organizations to move fast and distribute risk across many small releases.

In order for Continuous Delivery to work well there are a few ancillary requirements:

  • Release rollbacks must be instant and easy because sometimes things are going to break and getting back to a working state quickly must be painless and fast.
  • Patch releases must be able to make it from SCM to production (through a continuous delivery pipeline) in minutes.
  • Load balancers must be able to handle automatic switching between releases.
  • Database schema changes should be decoupled from app releases otherwise releases and rollbacks can be blocked.
  • App-tier servers should be stateless with state living in external data stores otherwise state will be frequently lost and/or inconsistent.

2005 = Persistent Servers / “Pray it never goes down”
2015 = Immutable Infrastructure / Ephemeral Servers

When a server crashed in 2005 stuff usually broke. Some used session replication and server affinity but sessions were lost and bringing up new instances usually took quite a bit of manual work. Often changes were made to production systems via SSH making it difficult to accurately reproduce a production environment. Logging was usually done to local disk making it hard to see what was going on across servers and load balancers.

Servers in 2015 are disposable, immutable, and ephemeral forcing us to plan for them to go down. Tools like Netflix’s Chaos Monkey randomly shut down servers to make sure we are preparing for crashes. Load balancers and management backplanes work together to start and stop new instances in an instant enabling rapid scaling both up and down. By being immutable we can no longer fix production issues by SSHing into a server but now environments are easily reproducible. Logging services route STDOUT to an external service enabling us to see the log stream in real time, across the whole system.

2005 = Ops Team
2015 = DevOps

In 2005 there was a team that would take your WAR file (or other deployable artifact) and be responsible for deploying it, managing it, and monitoring it. This was nice because developers didn’t have to wear pagers but ultimately the Ops team often couldn’t do much if there was a production issue at 3am. The biggest downside of this was that Ops became all about risk mitigation causing a tremendous slowdown in software delivery.

Modern technical organizations of all sizes are ditching the Ops velocity killer and making developers responsible for the stuff they put into production. Services like New Relic, VictorOps, and Slack help developers stay on top of their new operational responsibilities. The DevOps culture also directly incentivizes devs not to deploy things that will end up waking them or a team member up at 3am. A core indicator of a DevOps culture is whether a new team member can get code to production on their first day. Doing that one thing right means doing so many other things right, like:

  • 3 Step Dev Setup: Provision the system, Checkout the code, and Run the App
  • SCM / Team Review (e.g. GitHub Flow)
  • Continuous Integration & Continuous Deployment / Delivery
  • Monitoring and Notifications

DevOps can sound very scary to traditional enterprise developers like myself. But from experience I can attest that wearing a pager (metaphorically) and assuming the direct risk of my deployments has made me a much better developer. The quality of my code and my feelings of fulfillment have increased with my new level of ownership over what is in production.

Learn More

I’ve just touched the surface of many of the deployment changes over the past 10 years but hopefully you now have a better understanding of some of the terminology you might be hearing at conferences and on blogs. For more details on these and related topics, check out The Twelve-Factor App and my blog Java Doesn’t Suck – You’re Just Using it Wrong. Let me know what you think!

Huge thanks to Jason Hand and Joe Kutner for reviewing this blog post.

Refactoring to Microservices

Right now there is a ton of hype and pushback around Microservices. Most of the current debate revolves around when Microservices make sense with smart people arguing all across the spectrum. As with all architectural topics the right answer is “it depends” so you should never blindly chose Microservices without understanding your goals and how they align with Microservices.

Using the open source WebJars project as an example I’d like to walk through a process of deciding where to use Microservices and then refactor part of the webjars.org app to a Microservice. First a little background on WebJars… WebJars are JavaScript & CSS libraries packaged into Jar files and published on Maven Central for easy consumption by JVM build tools. The webjars.org site is a Play Framework + Scala app that provides search, publishing, and file service for the jsDelivr CDN.

Here is my checklist for determining whether a piece of functionality should be broken out into a separate Microservice:

  1. The piece of functionality does NOT have shared mutable state.
    When using a Microservice a copy of the data will be shared. Mutating that copy will likely not propagate those changes back to the original source and all of the other possible copies of the data. While shared mutable state is common in many OO apps, this makes it very hard to switch to Microservices. Functional Programming on the other hand encourages immutable data which makes it much easier to switch to Microservices where copies can be mutated but it is clear that those mutations do not act on the original or other copies.
  2. The piece of functionality has independent operational or computational needs.
    If SLAs, scaling, or deployment needs vary between different pieces of functionality then Microservices might make sense. For example, if one piece of a system requires five nines but rarely changes while another piece does not have an SLA requirement and changes multiple times a day, Microservices make sense. Likewise you shouldn’t need to scale up every part of a system just because one piece of functionality has significant computation needs.
  3. The piece of functionality has cross-platform clients.
    While sharing code across platforms (e.g. JVM, Ruby, Node.js, etc) is sometimes possible, it is often easier and more maintainable to just expose the needed piece of functionality as a Microservice so that any platform can use it. For example, webjars.org uses a bower-as-a-service Microservice that runs in Node.js because it uses the Bower NPM package. The webjars.org app is a cross-platform (JVM) client to the Node.js Microservice.

The whole webjars.org app is functional and uses immutable data so there isn’t any shared mutable state that would make it hard to break pieces of functionality out into Microservices. In Play Framework a controller is really just a stateless function that takes a request and returns a response. This means that any of the web endpoints can be easily moved without impacting the system.

One possible candidate for a Microservice in webjars.org is a utility that converts SemVer-style version ranges to Maven-style version ranges. The SemVer.convertSemVerToMaven() function is not side-effecting so it could easily become a Microservice. But at this time the utility does not have independent operational or computation needs and it also does not have any other clients than the webjars.org app. If the functionality was needed outside of webjars.org then it could easily be turned into a library but a Microservice would definitely be overkill.

Another candidate for a Microservice in webjars.org is a web endpoint that serves a file from a WebJar. The Application.file controller function is stateless and does not use shared mutable state so it could easily become a Microservice. This function is what provides the content for WebJars on the jsDelivr CDN. When a request for a WebJar file on jsDelivr is received, if the CDN does not have the asset it gets it from webjars.org. For example:
https://cdn.jsdelivr.net/webjars/org.webjars/jquery/2.1.0/jquery.js
Is backed by:
https://webjars.herokuapp.com/files/org.webjars/jquery/2.1.0/jquery.js

The operational and computational needs of this piece of functionality are pretty different from the rest of the webjars.org app. Let’s compare the needs:

The webjars.org File Service Rest of webjars.org
SLA If it goes down then many production sites break No production uptime requirements
Scaling Most load is handled by the CDN but sometimes load spikes when caches are stale or invalidated Very light load
Deployment Rarely changes Changes a few times a week

So this seems like a great candidate for a Microservice! Here are the steps I used to break out this functionality into a Microservice.

Step 1) Create a new Play + Scala app

I used Typesafe Activator to create a new Play Framework + Scala app:

activator new webjars-file-service play-scala

Here is the commit from that starting place:
https://github.com/webjars/webjars-file-service/commit/4ef3567a8ea901c31cc1060af67ed80331b9b6a4

Step 2) Clean up the build and copy the code into the new project

I copy and pasted the parts of the code that I wanted to move to the Microservice into the new project. Here is the full change set:
https://github.com/webjars/webjars-file-service/commit/5cc050f0dab16e8cb14296d811a965df9616ca0b

There was very minimal refactoring between the original source and the new Microservice. Everything worked great locally so it was time to deploy the Microservice.

Step 3) Create a new Heroku app and setup GitHub auto-deployment

I created a new app on Heroku:

heroku create webjars-file-service

Instead of doing the usual git push heroku master I setup auto-deployment so that whenever I push to GitHub, Heroku deploys the changes. Check out a screencast of how to do that:
https://www.youtube.com/watch?v=QUvxrzINj5Q

Now that the webjars-file-service is deployed let’s try it out:
https://webjars-file-service.herokuapp.com/files/org.webjars/jquery/2.1.0/jquery.js

Everything is working great so lets switch webjars.org over to the new Microservice.

Step 4) Make the webjars.org app use the new Microservice

To make webjars.org use the new Microservice I removed the actual logic but I didn’t want to break any clients that were using the endpoints. To do this I added redirects for the actual file service functionality and for the file listing functionality I added a utility that wraps the new webjars-file-service Microservice. Along the way I had to do a small refactor of some Memcache-related functionality. Here is the full change set:
https://github.com/webjars/webjars/commit/5b7cd4b8fa410cd6e4ef9824a69d28790f42e9bd

After pushing the changes to GitHub, Codeship verified that the tests passed, and Heroku deployed webjars.org.

This whole process only took a few hours and so far everything has been working great! Because the file service functionality in webjars.org did not have shared mutable state it was incredibly easy to move to a Microservice which enables me to handle it’s unique operational and computational needs.

The decision to move something to a Microservice is always full of “it depends” factors. Microservices are certainly not a silver bullet especially when dealing with code bases that have shared mutable state. Like any tool, Microservice can be a powerful way to help you, or they can be the chainsaw you use to cut down the tree that falls on you. Handle with care!