Architecture of an Open Source Digital Platform
Published: 2022-01-18 17:48:20

## **Introduction**
This is the  third  text about platforms. Now the topic addressed will be architecture and technologies that will compose the solution we want to provide and the ability to put the concept into practice. Finally, we will  elaborate  a  technical architecture  as an example. If you haven't read the  [second article](https://dev.to/lucasnscr/principios-de-uma-plataforma-digital-2d8e) , I recommend reading it because in this second text we define the principles that we will present in our architecture.

Now that we have a more detailed understanding of the topic, we are going to break down the specific layers of a digital platform, for that we are going to apply each step of this model with open source software, so we can be more generalists without being tied to cloud computing products or providers. , focusing only on architecture, technology, benefits and the responsibility that each division will have within this universe. The ideal of opting for this way of doing is that it allows us the creative freedom to use what is most convenient to your scenario, considering that the loose coupling allows us to implement and implement the steps mentioned later in this text of independently, remembering that if you want to take this build to your production, it is recommended to get commercial support.

## **Digital Platform and its Architecture**
After we conceptually understand what a platform is, purposes and what is necessarily important to base itself on this concept, we will carry out a technical approach where we will define premises. Some points are extremely important to be ratified, now so that the process of understanding the architecture is as simple and rich as possible. Many keywords composed the first texts, such as:  **Time to Market**,  **Low coupling**  and  **Performance** . When we take the words above into a technical context, we identify essential points of the architecture which are:  **Microservices** ,  **Container**  and  **Agility**. These three characteristics will be the basis of our construction, we will follow **Microservices patterns** , they will be deployed in a container and we will follow. Microservices patterns , they will be deployed in a container and we will follow the flow of agility as the delivery center.


![Platform Architecture](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fr9xai4yw8qslv1091v4.png)

Now that we have an image that illustrates our  **architecture** , we'd like to approach the flow from some perspectives. From the user 's point of view  , we illustrate three channels that can be an entry point for an architecture, be it  **mobile**, **web** or  **partners**. An important point is that there are several single responsibility components that interact with other components through communication standards, an important point is that removing one component  would not make  the platform as a whole unfeasible. From another perspective, but we see that we have an  infrastructure layer that supports the **container and orchestration layer**, these last two are managed through the **Devops structure** and **automation tools**  to enable both stages.

## **Legacy, Proprietary, and SaaS Tier**
Representation of what  **already exists in the operation**  within the business before the platforming process. In this stage  , **databases, messages and procedural tools for internal use and legacy systems** are applied . Generally, the characteristic of this layer is that  **they do not follow the architectural pattern** that we have defined here, but they are systems that already make up the company and that  **their migration processes can be extremely difficul**t  and what we are left with is the interaction with them. Aiming that a **reformulation of this layer goes beyond the objective we want to achieve** , we can define that this layer  communicates  with our platform. There are two points at this point that we need to mention, the **communication** and the  **Anti Corruption Layer**.

## **Communication**
Communication needs to be obtained **directly**, a good part of tools that position themselves as legacy have integrations with **HTTP**, this form of communication facilitates working together with our application layer.

## **Anticorruption Layer**
Layer that aggregates a set of services that build a new layer responsible for  the **translation** between the **semantics** of two systems. Usually composed of combinations of the Creation Pattern: **Facade** and  **Adapter**. When applied, a communication and data transaction mechanism is created. Facade **simplifies** access to a system where its functionality is exposed and its domain and implementations are hidden. The Adapter serves as a  **wraper** that has its responsibility activated, when a client sends a message to the Adapter and it **converts the message** equivalent to what the legacy application will understand.

When well built, this layer can have fundamental resources added to it for operation, which are:  security, service governance  and  monitoring.


![Representation for Anticorruption Layer](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/npm45l7aj68gc2qkf45m.png)

## **Application layer**
Main layer of our architecture, we will divide it into a few steps so that the **understanding** and **responsibility** of each component are well defined and facilitates understanding.

## **Application Microservices**
**Usually in this stage the core of the business** is concentrated , most of the innovation concepts will be applied there , after all this layer has as a prerequisite: **Development**, **tests and debug independently**, **Independent deployment** , **Independent scalability**, **Independent CI/CD**, **execution in containers** and **loosely coupled**.

The recommended development for this layer is the use of **agile frameworks** and made by **small teams** whose main mission will be the **end to end work**:  **Solution**, **Development**, **Test** and **Deployment**.

## **Microservices Integration**
After  core business **microservices** are implemented , these services need to integrate with their consumers through an **API**. At this point we need to apply an **API manager**. The main functionalities of the business **cannot be exposed to the end customer**, because for its operation it may require data from several other microservices to build the information. When requesting other information and accessing it, we need **another layer in the backend** that can be implemented as **another set of microservices**, this is where the **integration Microservices comes in**. This type of service can be similar to the other types of Microservices mentioned in the text, but its purpose is totally the  common **integrations** that exist in every system. The characteristics of this type of service are: **Service Orchestration**, **Data Wraper**, **Conditional Routing**, **Data Integration and Cloud Integration**. This set of services is closer to the application's end-user, they can also connect with legacy services by **HTTP** or the **Anti Corruption Layer**.

## **Microgateway**
After the construction of the Microservices layer, be it  application or integration, these services need to be **controlled,  protected  and  monitored**. To get this full range of features we need to apply a **centralized multifunctional component** to meet our demands. This component, in addition to guaranteeing what we have already mentioned, needs **to expose our functionalities to consumers**, and  this is where the Micro gateways come in . When we apply a set of micro gateways to a set of microservices we want to ensure **resiliency**, and that doesn't affect performance  or  availability  in our business which is made up of a set of microservices. Micro gateways provide the following functionalities: **Security (authentication and authorization), Throttling, rate limit, Monitoring, Analytics,  Cache and Monetization**.

## **Service Mesh**
When we think of a **cluster**, we think that it is made up of  **hundreds  of  micro components** that are running on it. Managing this from a manual perspective is impractical when thinking about **scalable operations**. Service  Mesh  arrives to  **facilitate the management** of this cluster. Via service mesh we can **control the traffic flow**  between application and integration services, we have the availability of the **control pane**  that helps us to control cluster configurations from the point of view of **data, security policies and telemetry**.

## **Governance**
At the moment we are generating a **digital platform** that is based on **Microservices**, we have **multiple components,  technologies and integrations**. In order for all of this to coexist with  **the organization, a governance** layer is needed  so that we can have  operational **control and visibility**. As we talk about teams and processes, in the architecture represented above there are several services that are implemented by  **different teams** that act autonomously. You need to know how to manage **SLAs, technologies, message format** among others. Having a governance layer makes it **viable to maintain** and **evolve documentation** that is critical to **expanding the operation**. Governance needs to be assigned with the following profile: **Documentation, Assessments, Lifecycle, Reviews, Forums and Comments**. Talking about governance involves a very plural theme, and can extend the theme to runtime, infrastructure and even design.

## **Monitoring and Analytics**
Starting this paragraph by highlighting both **mechanisms**, but let it be very clear that **monitoring and analysis are distinct requirements that ultimately communicate to extract value for different teams within an organization**. Monitoring is **essential** in the type of architecture we adopt, we need to have our applications being highly monitored from both a **business and operational** point of view  . When something is happening out of the box, we need to have **traceability**, only then can we be assertive in **solving problems** that we have tried to mitigate so far anyway, but they can happen. There are some items that are fundamental for a rich monitoring, which are the monitoring of **computational resources** and the **application monitoring**.

Computational monitoring is more linked to **infrastructure** and is based on monitoring items such as: **CPU, Memory, Threads and the  Garbage Collector**.

Application monitoring is more focused  on system **operation** and it is necessary to observe: **TPS, Response Time ,  Availability, Usage and Tracking**.

Sometimes monitoring is not enough to reach the highest level of availability and operational quality, depending on the business,  **95% availability is still something low**, for example  **financial operations**. **This is where analytics** comes in,  **analyzing deals transacting in real time is necessary to anticipate scenarios and make decisions**. Analytics can be divided as follows: **Analysis on stored data, Real-time analysis where alerts/notifications  are sent and immediate decision-making is required and Machine Learning**, this last point is essential so that crisis events are foreseen and that the operation is planned according to  the seasonal scale to  which the business is exposed.

## **Security**
Item present in any architecture definition, security is something more than desired is **necessary**. Being a **critical requirement** for any digital platform. Currently, the **greatest value an organization can have is its data** and protecting it is more than its obligation, it is its primary function,  Data is new Money . Here there are two essential points for security, which are: Authentication and Authorization.

**Authentication** is the verification of the identity of the respective user who is trying to access the functionalities of the system.

**Authorization** is the verification of the user's authority and whether the same is enabled to access certain resources or services.

Security is applied at the **micro gateway layer**, where it can interact with a **security provider** that validates user requests. **OAuth 2, OpenId and JWT** are the most popular and current technologies used for protection.

## **Devops and Automation**
After mentioning various topics like **scalabilit**y, **security**  and **operations**. The time has come to talk about, fundamental part of a digital platform, **Devops**. **Continuous Integration and Continuous Delivery**, are very important in performing an  **agile construction** of the platform, capable of allowing  **several daily releases** in productive environments, making your product very **competitive in the market**. Devops delivers a series of advantages that companies that still do not have this practice should adopt, which are: **Daily releases, Automated industrialization process, Improvement in the quality of deliveries and Governance**.

## **Loading Balancer**
Providing safe services to the world is an arduous task, Loading Balancer  can be instrumental in helping us in this activity. Knowing that yes, here **we have a point of failure** and that **we can mitigate that point of failure with multiple load balancer instances**. This layer takes care of applying  **additional security measures** that can be taken to protect the service from **malicious users** who want to attack the system .

## **Container Orchestration**
After structuring the entire development, it is necessary to select which **type of infrastructure** we will use. The selection of this infrastructure and the execution of the services plays a key role in the overall architecture solution. Microservices are intended to run inside **docker containers**, for example. The runtime of a platform, if applied in a container, demands other factors that are paramount, it is at this moment that we need to  **orchestrate containers**, its orchestration allows us to:  **Independent action in pod health regeneration, Automatic scaling, automated deployments and rollbacks and Service Discovery**.

## **Container Runtime**
The runtime of a  container is a central point for software, it is so fundamental that **this architecture as a whole is only viable because of them**, the container and its runtime. **By providing an  efficient engine**, we were able to **run multiple workloads on a single computing resource**. The container **hides the complexity of resource allocation** and I venture to say that it resembles the way an operating system handles activities, abstracting complexities and providing practicality with the following characteristics: **Platform independent software execution, Flexibility, Multiple environments and Runtime Stacking**.

## **Infrastructure**
The basis of the architecture is the infrastructure layer, it is our **hardware** that will allocate our software. As our architecture is **container-based**, we **can run them on any infrastructure**, **ranging from  physical machines to virtual machines and run by infrastructure providers (IaaS)**. The architecture is open and can be implemented with **multiple technologies for the same component**. Now let's apply  Open Source  in the architecture.

## **Open Source in Platform Architecture**
Below we have our architecture and we will describe the technologies used, remembering that they will all be OpenSource, following the indications of projects contained in **[Cloud Native Computing Foundation](https://landscape.cncf.io/)**.

**Infrastructure Layer**: In this topic we have several famous providers, such as **AWS, Google Cloud or Azure**. Here we will suggest the **[Open Stack](https://www.openstack.org/)** delivering an IaaS software, its main function is to create a virtualization layer over the resources of a local data center. 

**Container Runtime**: Here we have **[docker](https://www.docker.com/)** as the main tool, but docker despite being free for a single user, after a number of machines it requires a license, in this case we will recommend **[Rancher](https://rancher.com/products/rke)**, which is a project already recognized by the **CNCF**.

**Container Orchestration**: **[Kubernetes](https://kubernetes.io/)** is the adopted standard for orchestration. Most **IaaS** providers have integration with kubernetes, popularly known as **K8s**.

**Application Microservices**: To facilitate the implementation of application services, we will adopt **[Spring](https://spring.io/)**. The most famous **JVM** language framework is open source, contains a strong community and its knowledge is disseminated all over the world.

**Service Mesh**: This subject is largely dominated by **[Istio](https://istio.io/)**. Adopted by tech giants and its main focus is on performance and stability.

**Integration Microservices**: **[Apache Camel](https://camel.apache.org/)**, widely used for integration camel implements a series of integration patterns that can help the communication edges.

**Governance**: **[Umbrella](https://apiumbrella.io/)**, a leader in **API management** providing the following capabilities: call throttling, API keys, caching, real-time analytics, and availability of a profiling web interface.

**Monitoring and Analytics**: **[Prometheus](https://prometheus.io/)** is an open source monitoring framework that is capable of monitoring microservice architectures and **[Grafana](https://grafana.com/)** which is an open source platform for analytics and monitoring.

**Security**: **[Keycloack](https://www.keycloak.org/)**,sponsored by RedHat, the tool implements essential **authentication and authorization** items.

**Micro Gateway**: **[Kong](https://konghq.com/kong/)**, the most popular **gateway**, was developed to meet multi-cloud, its optimization for microservices and distributed architectures are fundamental factors for its choice.

**Loading Balancer**: **[Nginx](https://nginx.org/)** is one of the leading software load balancers that has an input controller implementation and great integration with kubernetes.

**Anti Corruption**: **[Apache Camel](https://camel.apache.org/)**, it implements a series of integration patterns, being one of the most adopted integration frameworks in corporations.

**Devops and Automations**: Here we can use **[GitHub](https://github.com/)** to be our source repository where we keep the source code produced. **[Jenkins](https://www.jenkins.io/)**, a widely used tool, can be used to automate processes allowing building automations that allow releases without human interactions. **[Ansible](https://www.ansible.com/)**, used to manage infrastructure and automate server deployments.