Expand and contract
By: Albert Starreveld
Tags: Canary Release, Database, Continuous Delivery, Blue Green Deployment, Progressive Delivery

Canary-releasing or zero-downtime/blue-green deployments are made easy by the tools and techniques we have today. Running a second instance of a service is a piece of cake. Especially in the cloud. But aren’t we forgetting something?

Canary-releasing and blue-green deployments have something in common: They require two versions of the software to be up and running at the same time.

It’s not just a matter of running multiple instances of software somewhere. You’ve got to think about backward compatibility, too! One of the instances will break if it’s dependency isn’t compatible.

What’s the most common shared dependency? The database! Blue-green deployments and canary-releases require your database to be backward-compatible. Use a pattern called Expand and Contract, or Parallel Change to get it sorted!

A practical example:

Assume we’re running a web store and the team is going to build the following story:

To receive my purchase when I’m not at home

As a customer with an office-job

I want to have my purchase sent to the office instead of my home address

Let’s assume there’s a database that contains a table with orders and the table already contains addresses. The current data structure looks like this:

Image 1.) The current data structure

Say we’ll solve the user story by storing two addresses instead of one: The delivery address and the billing address. Meaning the “old” address column in the database will be dropped, and two new columns will appear. This will be the new data structure:

Image 2.) The new data structure

The problem

These data structures aren’t compatible. The current version of the web store depends on the address field. Deploying the new data structure will drop that field, causing the current version of the web store to break.

As a result, neither blue-green deployments nor canary releases will work. It requires two versions of the same software are “up” simultaneously. In this case, the newer software is incompatible with the current database, and visa versa. If the “newer” database is “up”, the current version of the software will be “down”, and visa versa.

Backward-compatible schema changes

Implementing a new database schema in a backward-compatible way is done using multiple releases:

Image 3.) Changing a database-schema the backward-compatible way

Let’s apply it: We’re introducing the “BillingAddress” and the “DeliveryAddress” field, and we’re dropping the “Address” field.

Step 1.) Deploy the new schema

The first thing to do is to deploy the new schema. The old schema needs to remain intact and should be considered the source of truth. Assume the data in the new schema is incorrect or missing in Software Version B.

In the example of the billing address, that will result in the following data-structure:

Image 4.) Introducing the new columns, but leave the current data structure intact

Assume we’ve got canary releases. 10% of the users will be routed to the newest version of the software, 90% will be routed to the current version of the software.

In that case, for a given amount of time, Software Version A and Software Version B will be running simultaneously. Both versions will update the database simultaneously. As a result, 90% of the records in the new schema will not have (correct) values.

The version of the software that introduces the new schema should be writing to it, but not read from it.

Step 2.) Start using the new database schema

With the new schema in place, the next version of the software can start using it.

Software Version C has the same data structure Software Version B has. But there’s a difference: Version C depends on correct data in the new schema. Software Version C will consider the new database schema to be the source of truth.

If we don’t migrate the data from the old schema into the new schema in Release 2 (as shown in Image 3.), then Software Version C will not work. It depends on correct data in the new schema.

To make Software Version C compatible with Software Version B, Software Version C needs to keep the data in both the old- and the new schema up to date. Software Version B. does not “trust” the data from the new schema yet, so it depends on the data in the old schema.

Step 3.) Stop using the old schema

Next, focus on removing the old data structure. To get there, the first step is to stop using the field. As a result, this is what Software Version D looks like:

Image 5.) Stop using the old database fields

We can’t drop the columns from the old schema yet, because Software Version C needs to be able to keep it up to date.

Step 4.) Drop the old schema

After Software Version D stopped using the old database schema, it can finally be dropped. Software Version E will have the following data structure:

Image 6.) Drop the old fields

The downside

Using this method to change database schema’s is great. It makes things like Canary Releases possible. But there’s a downside.

Don’t use expand-and-contract if you’re not releasing often. Changing the database schema takes many releases. If you’re not releasing every month, a change to a schema takes half a year before you know it.

Summary

Modern progressive delivery depend on having multiple versions of the software “up” at the same time. To be able to do so, their dependencies need to be compatible. The database is the most common dependency.

Making changes to a database-schema can cause previous versions of the software to break. To prevent that, use a practice called expand and contract. Use multiple releases to create the new schema first, use it in the next release, stop using the previous schema in the next release, and drop it in the next.

Last, but not least: Release frequently!

Thanks to: