# Inter-Service Communication for Microservices

## Summary


In a monolithic application, all parts of the app access a shared database. Each part can easily invoke the functionality of another part. In a microservices architecture, an app is composed of many microservices, each potentially managing its own database. What happens if one service requires data or processing from another service? This is not as trivial or efficient as in a monolithic application.

Inter-service communication (ISC) is an important consideration when designing a microservices-based application. A badly designed app can result in a lot of communication among the different services, resulting in a *chatty app* or *chatty I/O*. Communication can be reduced by having fewer microservices but this replaces the monolith with smaller monoliths. The goal is therefore to achieve a balance by following good design principles and patterns.

## Discussion

### Why do microservices need to communicate with one another?

In the traditional approach to building monolithic applications, lot of communication was internal to the app. Such communication was often local, fast and easily manageable. When designing a microservices architecture, we break up the monolithic into independent parts, each of which has a well-defined role. While each microservice can be deployed and scaled independently, none of them deliver the full value of the application. A microservice will often require data managed by another, or require the services of another.

For example, consider a taxi booking application. Trip management and passenger management are separate microservices but a trip cannot be initiated without some knowledge or authentication of the passenger. Hence these two independent microservices, each doing its specific roles, will still need to communicate. 

While microservices architecture has brought benefits to build large-scale applications, it has also exposed the communication across microservices. Complexity that was previously hidden is now visible. Dozens or even hundreds of microservices that make up an app must be "wired together" properly to make the whole thing work.


### What are the different types of inter-service communication for microservices?

Broadly, there are two types: 

    + **Synchronous**: Client sends a request and waits for a response. Client code execution itself may prefer to receive the response via a callback (thread is not blocked) or wait (thread is blocked). Either way, the communication to the external world is synchronous.
    + **Asynchronous**: Client sends a request and doesn't wait for a response.With synchronous communication protocols, the receiver has to be available to send back the response. From application perspective, synchronous implies a less responsive user experience since we have to wait for the response. If one of the services in a chain of synchronous requests delays its response, the entire call flow gets delayed. 

With asynchronous communication protocols, the request (often called message) is typically sent to a message queue. Even if the receiver is not available, the message will remain in the queue and can be processed at a later time. Even if a service fails to respond, the original asynchronous call is not affected since it's not waiting for a response. 


### What protocols and data formats are suitable for inter-service communication for microservices?

**HTTP/HTTPS** is a synchronous protocol. Often service APIs are exposed as REST endpoints. **AMQP** and **MQTT** are examples of asynchronous protocols. To manage the queue, we can use RabbitMQ **message broker**. Instead of a message queue, we can also use an **event bus** for updating data asynchronously. 

Synchronous protocols are usually limited to one-to-one interactions. Asynchronous protocols have more options: **one-to-one** (notifications), **one-to-many** (publish/subscribe), or even allow for responses coming back asynchronously. For example, a user sends a tweet on her Twitter account that has many followers. This is an example of one-to-many publish/subscribe model.

The data that these protocols carry must be formatted in a manner understood by all. **Text-based formats** include JSON and XML. XML in particular is very verbose. Therefore, some implementations may prefer **binary formats**: MessagePack, Thrift, ProtoBuf, Avro. Note that these are well-known and popular formats and using them enables easier integration of microservices. It's also possible (but not preferred) to use a proprietary non-standard format internal to an application.


### What design patterns are available for inter-service communication for microservices?

Here are a few patterns to note:

    + **Saga Pattern**: A sequence of transactions, each one local to its database. A microservice triggers another via an event or message. If something fails, reverse operations undo the changes.
    + **API Composition**: Since table joins are not possible across databases, a dedicated service (or API gateway) coordinates the "joins", which are now at application layer rather than within the database.
    + **Command Query Responsibility Segregation (CQRS)**: Services keep materialized views based on data from multiple services. These views are updated via subscribed events. CQRS separates writes (commands) from the reads (queries).
    + **Event Sourcing**: Rather than store state, we store events. State may be computed from these events as desired. This is often used with CQRS: write events but derive states by replaying the events.
    + **Orchestration**: A central controller or orchestrator coordinates interactions across microservices. API Composition is a form of orchestration.
    + **Choreography**: Each microservice knows what to do when an event occurs, which are posted on an event stream/bus.
    + **Service Mesh**: Push application networking functions down to the infrastructure and not mix them with business logic.

### Could you compare orchestration and choreography?

Orchestration is a centralized approach. Calls are often synchronous: orchestrator calls service A, waits for response, then calls service B, and so on. This is good if service B depends on data from service A. However, if service A is down, service B can't be called. By coupling B with A, we've created a dependency. The orchestrator also becomes a single point of failure. 

Choreography enables peer-to-peer interactions without a centralized controller. It's more flexible and scalable than the orchestration approach. It's event-driven architecture applied to microservices. The logic of handling an event is built into the microservice. Choreography is asynchronous and non-blocking. The patterns CQRS and Event Sourcing are applicable to choreography. 

There are also hybrid approaches where a service orchestrates a few services while it interacts with others via an event stream. In another approach, an orchestrator emits events for other services and consumes response events asynchronously from the event stream for further processing. 

To conclude, 

> Orchestration is about control whereas choreography is about interactions.


### How do we handle service API calls that fail?

The simplest solution is to **retry** after a specified timeout. A maximum number of retries can be attempted. However, if the operation is not idempotent (that is, it changes application state), then retry is not a safe recovery method. 

The other approach is to use a **circuit breaker**. Many failed requests can result in a bottleneck. There's no point sending further requests. This is where we "open the circuit" to prevent further requests to a service that's not responding. 

We can also proactively reduce chances of failure by **load balancing** requests. A request must be processed by a service instance and we can select an instance that has less load. Container orchestrators (Kubernetes) or service meshes (Istio) enable this. 


### Are there any best practices for defining a service API?

Microservices must be designed to be independent of one another. One approach is to use **Domain-Driven Design**. This talks about understanding the problem space, using design patterns, and refactoring continuously. API should model the domain. It shouldn't leak internal implementations. 

APIs must have well-defined semantics and versioning schemes. A microservice can support multiple versions of an API or you could have a service for each version. Public APIs are usually REST over HTTP. Internal APIs can adopt RPC-style, where remote calls can look like local calls. However, they should be designed right to avoid chattiness. Consider the trade-off between making many I/O calls and retrieving too much data. 

Since application state is now distributed across microservices, design for and manage **Eventual Consistency**.  

While REST calls may use JSON, RPC calls can be more efficient with binary formats enabled by RPC frameworks such as gRPC, Apache Avro and Apache Thrift. To simplify API design and development, use an **Interface Definition Language (IDL)**. This will generate client code, serialization code and API documentation. 


### What are some anti-patterns to avoid when microservices need to communicate?

Sharing a database across many microservices is an anti-pattern since this introduces tighter coupling. A single data model for all microservices is another. Using synchronous protocols across many microservices increases latencies and makes your app brittle to failures. If microservices are not properly defined, this may result in chatty I/O that affects performance and responsiveness. 

An application may depend on hundreds of shared libraries. In the spirit of code reuse, all microservices may be relying on these libraries. This results in another anti-pattern called **distributed monolith**. Reusing code within a domain or service boundary is fine but anything beyond that is coupling. This form of coupling is worse than code duplication. Shared libraries can be considered but not made mandatory in some areas: logging, tracing, routing. 

It's not required that every event should contain full data, particularly when consumers are going to use only some of it. Consider sending essential data and URLs pointing to additional data. To communicate, consider REST plus its alternatives such as messaging and event sourcing. 


### What tools can I use to implement inter-service communication for microservices?

Among the message brokers are RabbitMQ, Kafka, ActiveMQ, and Kestrel. Cloud providers offer their own messaging systems such as Amazon SQS, Google Cloud Pub/Sub, and Firebase Cloud Messaging (FCM). Microsoft Azure offers Event Grid, Event Hubs and Service Bus for messaging. NATS, a CNCF project, is an open source messaging system.

Istio enables service mesh technology. Azure Service Fabric Mesh is an alternative. These rely on Envoy as the networking proxy. Similar proxies include Linkerd and Cilium. Conduit is a service mesh designed for Kubernetes. 

Netflix's Conductor can help with orchestration. For logging and monitoring, we have Retrace, Logstash, Graylog, and Jaeger. OpenTracing is an API that enables distributed tracing. Circuit breaker pattern can be implemented with Netflix's Hystrix. 

To define service APIs, Swagger can be used. In Java, REST-based microservices can be created with Spring Boot. 

## Milestones

2004

Eric Evans publishes *Domain-Driven Design: Tackling Complexity in the Heart of Software*. The book relates directly to object-oriented programming. Only later, it's relevance to microservices would be understood. 

2007

Michael Nygard explains the **circuit breaker** pattern in his book *Release It!: Design and Deploy Production-Ready Software*. This helps us build fault tolerant systems. In 2011, Netflix invents the Hystrix framework that includes the circuit breaker pattern. 

2009

Netflix embraces API-driven architecture that affects both development and operations. This is today seen as the birth of microservices. 

2010

At the start of this decade, the three-tier model (web tier, app tier, database tier) is found to break under heavy load. Microservices come to the rescue but they also bring problems relating to inter-service communication. Companies introduce libraries that are built into microservices to handle the networking aspects: Google's Stubby, Netflix's Hystrix, Twitter's Finagle. This however introduces coupling. A few years later these evolve to networking proxy and **service mesh**. 

Jan  
2016

Version 0.0.7 of **Linkerd** is open sourced on GitHub. It's based on Twitter's Finagle and Netty. This is one of the early beginnings of a service mesh. Likewise, **Istio** version 0.1.0 is released as an alpha version in May 2017.