Building Microservices - Fundamentals
Tags: microservices, systemdesign
Published: 2022-07-10 04:19:48

Over the years, applications have evolved from being a **Monolithic Centralised** deployments to **Distributed Microservices**.
If you are looking to _transition-into_ or _build-from-scratch_ a Microservice, Here some points to consider.

![system evolution towards microservices](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/h192fgcld1d2xv9j2295.png)

---

## Functional Design
After product requirements are finalised, we start identifying working parts of the system i.e. **functional components** which will eventually shape up as a service.

> ######_“If I only had an hour to chop down a tree, I would spend the first 45 minutes sharpening my axe.” – Abraham Lincoln_

If you have worked on Agile projects, you must have seen how requirements change from sprints to sprints specially during initial phases when product is not mature.

Now, Here are some of the task which would help in creating a solid functional design:

- Identifying **logical flow** of the system, what are the business requirements.
- Understanding **User interactions** with the system, is it through web/mobile interfaces or through API's.
- Knowing the **Data** flowing IN & OUT, should help in designing the database.
- Listing the input **validations** and business rules.
- Identifying **external dependencies** which needs to be integrated, could be in the form of _content, api, payment gateways_ etc.


######It may seem enticing to jump into code and infrastructure asap. With half-baked functional design, its going to be challenging and unproductive to code.

---


## Technical Design

### Identifying the Services 
Time to breakdown each functional component into one(_ideally_) or more(_edge-case_) smaller/independent services. Each service takes ownership of a specific functionality - **Separation of Concerns**.

###### Here's a food delivery app example, you can see how each service is catering to a **specific** need.
Services can communicate with each other or with external interfaces either through REST API's or Websockets, Synchronously/Asynchronously with Messaging systems like Kafka, RabbitMQ.

![food delivery app example of microservices](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/c1za1cxq8gq7j436ov6w.png)

######Quest to simplify & isolate the services shouldn't end up over-populating the system with a lot of services running - It only makes things difficult to monitor and manage.



### Technology Stack - _Pick & Move_
With **Service Oriented Architecture**(SOA), we always have a choice of kind of programming language to use for each service. As far as they are able to communicate through an efficient & secure channel, it's all good.

![Technology Stack](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/38xt0hs5rn6rabsdfewk.png)

As an example, 
Python for building an analytical service which requires ML. 
Java's async/multithreading for building compute intensive services like big data, video process.
Go with NodeJs for its non-blocking I/O capabilities, running multiple queries on MySQL or MongoDb at the same time keeping things simple and smooth.

![choice of programming language](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/a80qm9902j9y6higbvr7.png)


### Persistance 

_Separate database Vs Shared database_ - Let the debate begin  

![shared vs separate database](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6ji6ts05mfu3650wmzvs.png)

If we extrapolate the concept of loose coupling and distribution to databases, each service should end up owning a "database" which can be individually **scaled**. 

Imagine multiple teams working on shared database, how difficult it is to keep track of every change.

Separate databases are easier to migrate, you can fine tune each depending on requirement.

Also you get **Polyglot Persistence**, a fancy term  to choose a desired database as per requirement. 
For example: 
For unstructured data -> NoSQL. 
For co-related & structured dataset -> RDBMS 
For TextSearch -> ElasticSearch
 
### Messaging 
So now we have multiple services running separately, the question is how will they communicate with each other or with an external interfaces.

Service communication can broadly divided in to 
1. Synchronous - _real time_ 
2. Asynchronous - _not expecting immediate response_

**Synchronous** (Sync)
Happens in real time, client sends the request and waits for response. HTTP request/response is a good example of Sync messaging, when you send a http request you are waiting for a response  (200,404,400,500). If the response takes time, the client is blocked and eventually timeouts.

> ######They say Sync is an Anti-Pattern, communicating parties need to be aware of eachother - _What do you think, reply in the comment_.

**Asynchronous** (Async) 
With Async, communicating parties don't have to wait for server response and transactions to complete. If a service is unavailable, messages will be delivered when the services are up and running with help of **Message Brokers** who receive, store, route and deliver messages between different services.

[Apache Kafka](https://kafka.apache.org/) & [RabbitMQ](https://www.rabbitmq.com/) which have been quite popular Message Brokers .

### CI/CD Workflow
_The great Mono vs Multi Repo debate_  
When we are designing microservices, we should be thinking holistically. The loose coupling should we visible in almost all areas from coding to deployment.
Each service should be hosted in repository of its own and should be deployed through an isolated CI/CD pipeline which can be easily triggered from another upstream pipeline.

![Version & release](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/81aq1i7hn1n26y2022q8.png)

### Monitoring - _Do we need to discuss it_ 
Ah, a pain area for microservices. Monolithic systems are better off, as they are tightly coupled & more constrained. If anything goes wrong, it's easier to find and fix.

With Microservices, you never know what would go wrong and where. Lots of small parts, things are distributed, log needs to be read from multiple places.

So, We should be investing in a **Centralised logging** framework such as **ELK stack (Elasticsearch-Logstash-Kibana)** which can fetch logs from different services, accumulate it and you can view & analyse them at will.


**Summary**
We are building a system which can be developed, deployed and tested atomically. Each part is as loosely coupled as possible.

If you are in a dilemma - Should you go for it now, look months or years down the line.

Be careful of **choice overload** :  There are tons of design pattern, choice of programming language, what cloud services to got with and so on. The key is to leverage as much as you can - keep cost & timeline low.