# Design Patterns for Microservices and Containers

## Summary


For common software design problems, design patterns provide reusable solutions. Rather than design a solution from scratch, it's faster and easier to understand the context and see what design patterns can be reused for your application. 

For a cloud application that uses containers and microservices, there are patterns to do the same. Called either *Design Patterns for Microservices* or *Container Design Patterns*, we use these terms interchangeably in this article.

Let's note that this article is not about container data structures or components. In programming, containers are data structures such as sets and queues that contain other data members. Such containers have design patterns too.

## Discussion

### What's the motivation for adopting design patterns for microservices?

Sometimes there's a need to scale your monolithic application, to release updates faster or to manage large teams of diverse skills. One approach is to re-architect your application into a bunch of microservices that get deployed on the cloud as containers. This refactoring can be a long process for a complex product. It can also be daunting for folks new to the microservices architecture.

This is where design patterns can help. These patterns have evolved from best practices that engineers used when managing microservices and containers. When solutions to problems are known, they can be applied to similar problems in the future. Patterns formalize this concept of reusable solutions. 

Patterns help you to build your apps faster and make them more reliable. Moreover, you may able to use libraries or tools–Amalgam8, Hystrix, Eureka, Zuul –that have already implemented some of these patterns.


### What are the elements of a good container design pattern?

A design pattern must be clearly described with concrete examples and best practices. Without a clear understanding, others will not be able to apply or even know if it suits their application needs. The pattern must be agnostic of container runtimes or cloud infrastructure. 

For any pattern to be practical, it's necessary that the containers themselves are properly designed. Container design principles can be applied in two different contexts:  

    + **Build time**: single concern, self-containment, image immutability
    + **Runtime**: high observability, lifecycle conformance, process disposability, runtime confinementFrom the perspective of delivering software-as-a-service, similar principles have been noted in Heroku's The Twelve-Factor App. 


### What are the common design patterns for microservices?

There are dozens of design patterns for microservices that can be grouped by categories such as decomposition, observability, testing, UI, communication, database, security, deployment, and so on. Google engineers identified the following in their work with Kubernetes: 

    + **Single container management**: graceful termination, probing, config maps
    + **Single node multi container**: sidecar, ambassador, adapter
    + **Multi node**: leader election, work queue, scatter/gatherMicrosoft has published a number of design patterns including ambassador, backends for frontends, CQRS, event sourcing, gatekeeper, gateway aggregation, sidecar, strangler, throttling, etc. Other patterns to note are model, denormalizer, ingestor, service registry, edge controller, API gateway, circuit breaker, fallback, command, bounded context and failure as a use case. 


### Since there are so many patterns, which ones are important or essential to know?

It's hard to claim that one pattern is more important than another. It all depends on the context and the needs of a particular application. However, we can highlight the following patterns:

    + **API Gateway**: Clients call your application via command-line tools or GUI. Rather than call each service of your app separately, all calls will go via the gateway.
    + **Service Registry**: Since microservices are created and destroyed dynamically based on load, this pattern helps in connecting clients to available instances of microservices. Microservices register with the service registry.
    + **Circuit Breaker**: When a service is down, this failure could cascade to other services dependent on it. To prevent escalation, if failures exceed a certain threshold, services will not repeatedly call a failed service until recovery happens.
    + **Fallback**: Also called *Chain of Responsibility* pattern, if a service is unable to complete because of a dependency, a fallback method is called. By combining fallback pattern with circuit breaker pattern, more resilient apps can be built.

### Could you explain patterns for running two containers on a single node?

Let's look at three patterns that each involve two closely associated containers. It makes sense to pair containers within the same node (or same Kubernetes pod) if they have the same lifecycle. One of them is the main container while the other can be seen as a helper. We explain them as follows: 

    + **Sidecar**: To provide an auxiliary service to the main container, such as for logging, synchronizing services, or monitoring.
    + **Adapter**: To standardize or normalize the output/input/interface between the main container and the external world, such as reformatting logs or invoking libraries that don't have expected language bindings.
    + **Ambassador**: To connect the main container to the outside world, such as for proxying localhost connections to outside connections. For example, main container will always see a localhost database with the ambassador resolving this to the actual database outside.

### Are there patterns to manage data in a microservices architecture?

For easiest migration from a monolithic app, all services could share the same database. For better isolation and modularity, we can adopt private-tables-per-service and schema-per-service, perhaps on the same database server. For high throughput requirements, each service could have its own database server. 

One challenge with microservices is to ensure data consistency across all services. An important principle is to have a single authoritative source of truth: all others are read-only and non-authoritative. Services can synchronously query for data or asynchronously update it via events when that data changes. Whether to use synchronous API or asynchronous messaging depends on your application since both have their trade-offs. 

An event-driven architecture using publish-subscribe model aids asynchronous updates. Store only what a service needs. If services are constantly exchanging data, you might want to rethink service boundaries. 

A transaction that involves multiple services is not easy to implement. Some patterns are available: Saga Pattern, API Composition, Command Query Responsibility Segregation (CQRS), Scheduler Agent Supervisor, and Compensating Transaction. 

## Milestones

2003

To allow trace propagation through multiple microservices, the **Correlation ID** pattern is identified by Gregor Hohpe in his book *Enterprise Integration Patterns*.  It's important to note that the pattern is identified in a different context and only later applied to microservices.

2004

Influenced by fig trees, Martin Fowler documents the **Strangler** pattern. This is useful when migrating a monolithic app to a microservices architecture. Rather than have cut-over date to move from old to new, this pattern enables a gradual transition. 

2007

In his book *Release It!*, Michael Nygard introduces the **Circuit Breaker** pattern. With this pattern, a service "fails fast" by not calling another service that it knows is already down. Netflix's Hystrix framework implements this pattern. 

2011

The term **microservices** is first used at a workshop in Venice. By 2012, ideas and case studies are presented. Adrian Cockcroft of Netflix sees this as "fine-grained SOA". 

2013

Although the history of containers can be traced to UNIX of late 1970s, **Docker** as a container is released in 2013. Developers begin to see containers as a natural fit for microservices. 

2016

Brendan Burns and David Oppenheimer present a bunch of patterns for container-based apps at the HotCloud 2016 conference. This paper may be considered foundational and credited with putting the spotlight on container patterns.