How to get the most out of the cloud
By: Albert Starreveld
Tags: SaaS, Software Development, Iaas, Progressive Delivery, Cloud

Over the years, building and testing software that runs on physical servers has become the status quo. The database is the most important part of the application. Usually, it’s the only dependency an application has. Testing has become a no-brainer. Most of the tests are unit-tests and we run end-2-end tests manually or with Selenium to see if everything integrates properly.

Moving an application to the cloud is usually not too difficult. It will require some minor modifications. So why change anything to the way we build software?

1.) The difference between on-prem and cloud

In the end, every application runs on a computer. The cloud, and on-premise both depend on physical servers in a datacenter. The difference is who owns them, who’s responsible for maintenance, and who pays for them.

In the cloud, servers are shared amongst customers. On-premise, the company owns its hardware. On the bottom line that makes a difference:

The pricing model

Buying a server is a one-time investment. It’s a matter of buying the hardware, some operating system licenses, and probably a database license. That should be sufficient to run some applications for a couple of years.

To get the most out of the investment, the server will probably run as many applications as possible. And it will probably always be “up”. Shutting it down is probably more expensive than shutting it down. The only thing it needs is electricity, and that’s not expensive.

Cloud is different. The biggest difference is: You can pay as you go. Rent a server for a day, delete it when you’re done. You’ll be charged for a single day of usage, not for one physical machine.

Server maintenance

To host a website on a server, for example. First, you’ll need a server with an operating system. Then, you’ll need to install and configure Apache or IIS. They require an operating system.

In the Cloud, you don’t need a server to get a website up. Just purchase the application-runtime instead. You’ll get a place where your application runs without having to purchase and configure a server, nor do you have to patch, or secure the operating system. That’s done by the cloud-vendor. That saves a lot of time, resources, and money.

Cloud doesn’t mean you can’t use Virtual Machines anymore. If you want, you can get one. But typically, they’re more expensive than using cloud-native components.

2.) Getting the best out of the cloud

Moving to the cloud may be a matter of replacing the servers you have on-premise with virtual machines in the cloud. But a lift-and-shift approach has a down-side. It’s relatively expensive in the long run. Moving away from virtual machines to a more cloud-native approach creates opportunities and maybe cheaper.

Cloud-native applications aren’t just cheaper to host, they’re cheaper to build too. Using off-the-shelve components vastly shortens the time it takes to build an application and thus gives a business a competitive advantage.

Use the cloud to build software faster

Nobody builds software from scratch anymore. Many generic parts of an application are available as an NPM- or a NuGet package. Cloud takes that to the next level:

Software as a Service, the building blocks of a project

How many log-in screens have you built in your career? Finally, there’s no need to do so anymore. Such components are available as building blocks in the cloud. They provide user-management screens, MFA, and lots more. Auth0 and AD Connect offer this service for example. All you need to do is configure it, and wire it to your application.



Lots of functionality is available in the cloud. Functionality like logging in, sending e-mails and text messages, facial/object recognition APIs, Blockchain, billing, support-ticketing, you name it. Combining these components with open-source packages makes it possible to build an application with very few lines of code.

How many log-in screens have you built in your career? Finally, there’s no need to do so anymore. Such components are available as building blocks in the cloud. They provide user-management screens, MFA, and lots more. Auth0 and AD Connect offer this service for example. All you need to do is configure it, and wire it to your application. Lots of functionality is available in the cloud. Functionality like logging in, sending e-mails and text messages, facial/object recognition APIs, Blockchain, billing, support-ticketing, you name it. Combining these components with open-source packages makes it possible to build an application with very few lines of code. Infrastructure- and Platform as a Service and containers, the building blocks of the infrastructure

Setting up a SQL server or an Active Directory server is history. All of that is available as a service in the cloud or a Docker container.



A lot of products are available as a service or have been containerized. There are two ways of getting SQL Server up and running, for example. Either pull a Docker image and run it or get a managed instance on Azure. This applies to a lot of different services. As a result, often, there’s no need to configure, maintain, and pay for a virtual server anymore.

For a lot of applications, either the cloud or open-source provides the building-blocks of an application. That changes the way an application is built. It requires fewer lines of code to build a working application. And compared to on-premise code, a lot more code will invoke third-party APIs or store state somewhere.

Different services have different fees. In an early stage, it might be cheaper to store data in Azure Storage whereas when the amount of records increases, it might be more appealing to store (parts of) the data in a SQL Server instance. Applying patterns like Hexagonal/Ports-and-Adapters makes migrating easier.

The down-side

There’s a down-side, too. Your application has to be able to deal with services that are temporarily unavailable. Consuming SAAS products has an impact on the availability of the application. Most of the services have an uptime guarantee of 99.99%. But not every service will be down at the same moment. Imagine having an application that has a dependency on 3 services. The downtime is:

99.99% * 99.99% * 99.99% = 99.97% uptime guarantee

The more services are used, the lower the uptime guarantee gets. That’s why circuit breakers and Event-Driven Architecture is applied in cloud-native software.

Save money by not paying for idle resources

Imagine running a web-store. Usually, it has a couple of thousand visitors a day. On Black Friday, millions of customers visit the site. Hosting an application in the cloud could pay off in this scenario.

An on-premise configuration has a maximum capacity. The amount of hardware the business owns is probably an optimum between the cost of the hardware and the profit they will make with it. That might mean some potential customers are unable to visit the web-store on Black Friday because the server can’t handle the number of requests. It also means some of the servers the company owns have been purchased for the sole purpose of handling traffic on Black Friday. They’re idling for the rest of the year.

In the cloud, it’s possible to scale up and down, vertically and horizontally whenever you want. As a result, it’s possible to scale up to the capacity needed to serve the site to every potential customer on Black Friday, and scaling down the next day. That way, the company can serve the site to every customer without having to pay for that capacity for the rest of the year.

Horizontal scaling

Just like on-premise, there’s a maximum to the vertical scale of a resource in the cloud. Under the bonnet, there’s always a physical server that has a limited number of CPUs.

The difference is in scaling horizontally. Cloud vendors have a vast amount of hardware available, allowing clients to scale horizontally, almost endlessly. An application doesn’t scale horizontally out of the box. It needs to be built in a certain way. How to, depends on the nature of an application, but commonly, APIs are made stateless, and read-, and write operations are separated with the CQ(R)S pattern. These patterns make it possible to use replica’s of the data sources for querying, decreasing the number of operations the main data source needs to process. To ensure every request can be processed, write-operations are queued.

Relational databases don’t scale well. Consider using a NoSQL database instead.

Improve user experience by minimizing the impact of deployments

Imagine running an international webshop. Typically, when you’re deploying a new version, it’s a good practice to take the current version offline when the newer version is up and running. But how do you know it’s up? Is it up when a given endpoint returns a 200 OK? Or is it up if customers keep ordering products on the webshop?

Being able to run multiple versions of the same software makes it possible to serve the newer version of the software to a small number of end-users. Undo the deployment automatically if the newer version causes issues that haven’t been identified during the testing process. Monitoring the conversion-points on the website makes will tell if the newer version of the software is still functioning correctly. In the example of the webshop, an indicator could be: Are people still ordering products? If they aren’t, roll back the deployment!

These concepts (blue-green, ring-based, and canary releasing) have been around for some time. They aren’t cloud-specific concepts, but they’re easy to apply in the cloud and they minimize the impact of deployments. As a result, potential issues have a much smaller impact on the business.

3.) How it impacts the software and the tests

To build an application faster, to deploy it smarter, and to scale it more efficiently, the software can’t be built the same way on-premise oriented applications are built. To get the most out of the cloud, it needs to have the following characteristics:

It needs to be backward compatible, to support running different versions simultaneously.

Read- and write operations need to be separated to support horizontal scaling.

APIs need to be stateless to support horizontal scaling.

The application needs to cope with the fact that parts of the infrastructure may be temporarily unavailable.

The tests need to be quick enough to support deploying multiple versions of the software a day

Common patterns

On-premise, when a server is down, every application on the server is down. With cloud-native applications, that’s not the case anymore. Probably, only a little part of the infrastructure will be down. When the application is built right, just a little part of it is impacted. And by having multiple small applications, instead of one giant application, the impact of an outage may even smaller.

Assume we’re still running a webshop and there’s a database outage. Normally, it would be impossible for a customer to complete his order in that case. It can’t be stored in the database. Event-Driven Architecture makes the application more robust. Event-Driven applications treat the event of a customer completing his order is an “event”, which is stored in a queue. If the database is down, the event can’t be processed and will be saved so it can be retried later.

There are several ways to build an Event-Driven application. Commonly, Command Query Responsibility Segregation is applied, meaning one part of the application is responsible for processing all write operations, and another is responsible for handling all read operations. The read operations use a separate, dedicated read-model; a queryable representation of the main data source of the application.

The write-part of CQRS works with events, event-handlers, commands, and command-handlers. By increasing the number of nodes that process these commands and events, by introducing replicas of the read-model, and by increasing the number of nodes that process the read operations, the application scales horizontally, endlessly.

The tricky part is to model the commands and events correctly. Often, people use Domain-Driven Design to create the model.

Continuous Delivery? Progressive delivery!

Backward compatibility doesn’t have that big an impact on the software itself. It impacts the software delivery process. The database is usually the dependency that needs to be backward compatible. Removing a column isn’t a matter of just dropping it anymore. When two different versions are used simultaneously, they write data to the same data source differently. The software needs to cope with that. As a result, it takes quite some releases to change a data-schema.

To change a database in a backward-compatible way, use a method called Expand and Contract or Parallel Change and approach releasing the software differently.

Typically, a release contains one or more features. With Expand and Contract, it takes five releases to change a database-schema. As a result, it takes at least five releases to create a new feature. In other words:

Release != feature

Having backward-compatible versions of the software makes it possible to do:

Canary releasing

Bring a new version to production and introduce it to a small group before deploying it to all of the end-users. Get real user feedback and be sure the new release doesn’t cause trouble.

Bring a new version to production and introduce it to a small group before deploying it to all of the end-users. Get real user feedback and be sure the new release doesn’t cause trouble. Blue-green deployments

Deploy new versions of the software anytime a day without causing downtime.

Deploy new versions of the software anytime a day without causing downtime. Dark-launching

Ship new functionality to a small group of end-user and get real user feedback.

Testing

In many cases, an end-2-end-test requires a system to be in a given state. To get it in that state, usually, database scripts populate the database with test data.

Having multiple, smaller applications, which all have their own (type of) data-source, makes end-2-end testing very complicated. Every individual microservice needs to be set-up differently.

And cutting big application into small ones, and deploying it frequently introduces a problem: Versions… Every environment will have different versions of the software all the time. And if it has the correct version today, it’s will be replaced with a newer one tomorrow.

To cope with that, the test suite needs to have the following characteristics:

Daily releases are impossible if the testing process takes longer than a day. The faster the test-suite, the better. The test-suite needs to be able to run in less than half a day.

Maintenance of the test cannot take longer than half a day.

They can guarantee working software, end to end, in every environment (even in production)

Maintenance to an end-2-end test makes changing the software fast, and frequently they’re too complex. There’s a new alternative: Contract testing. As a result, every individual application gets its testing pyramid which looks like this:

Being able to run different versions of the software opens another door. Testing changes into a process you do in production, too. By being able to run different versions of the same software it’s now possible to:

Do dark launches

Do A/B testing

Ring based deployments

Canary releasing

And so forth…

Summary

Cloud is more than server virtualization. In fact, for many applications, there’s no need to have a server at all. Cloud vendors offer infrastructure as a service. All you need to do is select the components the application needs, and the cloud vendor supplies it. It’s a lot cheaper, too.

Because you’re not owning any hardware, when you’re in the cloud, it’s possible to scale up, and down, whenever need be. That way, you can always get the capacity you need at a particular moment.

The cloud offers software as a service, too. Building your own log-in screen may be a thing of the past. Selecting the components you need to build your application, makes building the software a lot simpler.

Build applications differently to exploit the possibilities of the cloud.

Use Event-Driven Architecture and CQRS to be able to scale horizontally

Apply Domain-Driven Design to cut your monolith into little pieces

Stop releasing when a feature is done, release all the time

Do not use E2E testing, use Contract Testing instead

What to do next

Applying Domain-Driven Design, CQRS and the testing pyarmid is complex. Read how to, and find example code in the following articles: