FastAPI Microservice Patterns: Service discovery in container orchestration platforms
By: Florian Kromer
Tags: Python, Skaffold, Fastapi, Design Patterns, Microservices

FastAPI Microservice Patterns: Service discovery in container orchestration platforms

What’s the problem?

To enable communication between services based on synchronous communication mechanisms like REST or gRPC the services need to know each others location (IP address and port). But how can a client (service which depends on an endpoint of another service) discover the location of the service it depends on?

Solutions alternatives

There are two patterns which can be made us of to solve this problem. The client-side discovery pattern or the server-side discovery pattern. Both approaches have advantages and disadvantages. Deciding when which pattern is the right one is not in the scope of this post.

One solution: Server-side discovery pattern

This post is about the server-side discovery pattern which can be implemented in various ways. In this post the pattern is implemented in combination with the service instance per container pattern and the service deployment platform pattern, service registry pattern → 3rd Party Registration pattern.

The different patterns solve different problems:

server-side discovery pattern : How does the client of a service discover the location of a service instance?

: How does the client of a service discover the location of a service instance? service instance per container: How are services packaged and deployed? (Packaged for and deployed as container, deployed as function in a server-less platform, packaged and deployed for VM?)

service deployment platform pattern: How are services packaged and deployed? (container orchestration platform, server-less platform or PaaS?)

service registry pattern: How do clients of a service and routers know about the available instances of a service?

3rd party registration pattern: How are service instances registered with and unregistered from the service registry?

The service instance per container pattern is implemented using Docker which builds a container image for the service. The service deployment platform pattern, the server-side discovery pattern and well as service registry pattern -> 3rd party registration pattern are implemented using Kubernetes.

The service deployment platform pattern is implemented by using Kubernetes which deploys services as part of pods. “Pods are the smallest deploy-able units of computing that you can create and manage in Kubernetes”. Pods run either a single container or a several containers that need to work together. Kubernetes uses the container image of the service to create a container instance of it which is run as part of the pod. Kubernetes also handles the routing and service registration which implements service registry pattern -> 3rd party registration pattern. All incoming requests by the client go to a router. The router queries the service registry which location the service instance has. Usually the router acts as load balancer as well to increase service availability by forwarding client requests to a service instance which is “not doing work ATM”. For a single service Kubernetes manages usually several service instances in several pods.

As Kubernetes is widely used for container based service deployment this implementation should be adaptable to the majority of on-premise deployment use cases. In case of cloud deployments (service deployment platform pattern implemented with PaaS) there are provider specific services like e.g. AWS Elastic Load Balancing or Azure Load Balancer which are used to implement server-side service discovery instead.

Pattern implementation

The post FastAPI Microservice Patterns: Local Development Environment — Skaffold, docker, kubectl and minikube in a nutshell describes how to setup the local development environment and how to get the source code of the pattern implementations. The sub directory server-side-discovery of the source code repository contains the example implementation of the pattern. The logic does not make sense but makes it obvious how the pattern solves the problem of how the client of a service discovers the location of a service instance. A service-b provides an endpoint /answer implemented in service-b/app/main.py

@app.get(“/answer”)

def ultimate_answer():

return {“answer”: 42}

which is called by the endpoint /ask of service-a :

@app.post(“/ask”)

def request_answer():

endpoint_of_service_b = “http://service-b/answer"

response_of_service_b = requests.get(endpoint_of_service_b)

json_response_of_service_b = response_of_service_b.json()

return {“answer via service-b”: json_response_of_service_b[“answer”]}

The important point here is that service-a does not have to know the IP address and the port of service-b . Kubernetes makes service-b available to service-a via http://service-b . service-a can access http://service-b/answer to access the corresponding endpoint.