⤴️ Top 6 projects for building scalable cloud-based applications 🚀
Tags: webdev, programming, tutorial, devops
Published: 2023-10-30 14:39:46

## TL;DR

This article lists 6 top projects for simplifying the deployment and building scalable cloud-based applications for developers. 🎉

Feel free to explore these projects, contribute to your favorites, and support them by starring their repositories.🌟


![mylist](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ncmz12tyxu79tzj5xgfn.gif)

<hr />

## Odigos - Open-source Distributed Tracing

**Monitor all your apps simultaneously without writing a single line of code!**
Simplify OpenTelemetry complexity with the only platform that can generate distributed tracing across all your applications.

We are really just starting out.
Can you help us with a star? Plz? 😽

https://github.com/keyval-dev/odigos

[![GiveStar](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/k6l36g0qcds52ww1do96.png)](https://github.com/keyval-dev/odigos)

<hr />

### **1. [OpenTelemetry](https://opentelemetry.io/)**

> 💡 Collect telemetry data and forward it to analysis tools.

![OpenTelemetry](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hjnuu0w2fgeiwyadtjz3.png)

OpenTelemetry is an observability toolkit that helps us collect data like **traces**, **metrics**, and **logs**. It is vendor and tool-neutral 🛠️ meaning it can work with different Observability Backends.

> What OpenTelemetry is not? 🫣

OpenTelemetry is not a monitoring backend. Many people confuse themselves between OpenTelemetry and Observability backends like **Jaegar**, **Datadog**, etc. OpenTelemetry only generates, collects, and exports telemetry data. Storage and visualization are left to other tools. 📊

OpenTelemetry sits between the code and the observability tools we use. This provides us with flexibility and makes it easier to switch tools. 🔄

> To better understand how _OTel_ (short for OpenTelemetry) works,  take a look at the below image. 👇🏻

![Working of OpenTelemetry](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/bkupgifpjknk8nxsf1up.png)

{% cta https://github.com/open-telemetry
 %} 🌟 OpenTelemetry on GitHub {% endcta %}

***

### **2. [Kubernetes](https://kubernetes.io/)**

> 💡 Container orchestration platform for managing applications.

![Kubernetes](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/h8u6iwfpujvhrd10lt2r.png)

Kubernetes is the most important repo on the list when we are talking about cloud-native applications. How, can we skip Kubernetes right? 😜 It is the most famous orchestration platform for container-based applications and a foundation for many cloud-native tools to work.

> **What is Container Orchestration, and why do we even need it? 🤔**

A container engine is sufficient for managing a few containers, but for hundreds or thousands of containers and hundreds of services running, the complexity grows huge 📈. Trust me, it is something I have had experience with. 🥲

This is where the orchestration engine like **Kubernetes** comes into play.

The best way I think of Kubernetes is as a **puppet master** 🎭 for containers, manipulating their actions, coordinating their tasks, and ensuring they work in harmony across a cluster of servers.

> Refer to the image below to get a better understanding of what I meant by puppet master.

![Working of Kubernetes](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/s0jmccrhedqwia8f3f2m.jpg)

{% cta https://github.com/kubernetes/kubernetes
 %} 🌟 Kubernetes on GitHub {% endcta %}

***

### **3. [Backstage](https://backstage.io/)**

> 💡 Unifies tools for a streamlined development environment.

![Backstage Dev Portal](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kpszeeza08pnx1xlmtxe.png)

Backstage is an open-source project that serves as a developer portal, offering a unified view of infrastructure resources, CI/CD pipelines, and observability metrics. 📊 It gives us all the building blocks to create a platform ourselves that manages our complex software development ecosystem.

It provides us with easy access to essential information without requiring in-depth tool knowledge 🧰, acting as a central hub 🏛️ for company-wide development needs.

The core functionalities of Backstage are:

- **Service Catalog 📋:** This is the core Backstage **foundation**. It allows us to track and manage all our services, apps, and pipelines in one place.

<br />

- **Software Templates 📄:** With the Backstage UI interface, we can easily create new services with just a few clicks or use a **YAML** template to add necessary resources.

<br />

- **Tech Documentations 📃:** It adds seamless technical documentation management, ensuring teams can effortlessly keep content up-to-date, all from their code repositories.

> In short, this abstracts away a lot of **CI/CD** and provides us with information on the company's services. 🚀 I would say this is a **must-have** tool for any company that is planning to upscale and streamline their development operations.

{% cta https://github.com/backstage/backstage
 %} 🌟 Backstage on GitHub {% endcta %}

***

### **4. [Prometheus](https://prometheus.io/)**

> 💡 System monitoring tool with time-series data.

![Prometheus Monitoring Tool](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/i6wsdw0itsb8ymn62b67.png)

Prometheus is a free open-source software application used for event monitoring and alerting. 🧐 It records real-time metrics in a time series database (allows querying the data using its own query language called **PromQL**).

It also provides an alerting system that allows us to define thresholds and conditions for alerts.

When certain metrics exceed or fall below defined thresholds, Prometheus can trigger notifications, helping in successfully addressing potential issues before they impact the system's health.

> In short, Prometheus helps keep a close watch 👀 on our infrastructure's health and performance.

{% cta https://github.com/prometheus/prometheus
 %} 🌟 Prometheus on GitHub {% endcta %}

***

**🔴 GOOD TO KNOW:** Notice that I've mentioned two similar projects, _OpenTelemetry_ and _Prometheus_, which serve similar purposes. 🔄

Prometheus is primarily a system-level monitoring and alerting tool, while OpenTelemetry is focused on collecting and forwarding application-level telemetry data.

Prometheus didn't initially support OpenTelemetry metrics, but now they can work together.

> For more details, you can refer to this [article](https://horovits.medium.com/prometheus-now-supports-opentelemetry-metrics-83f85878e46a).

***

### **5. [Cilium](https://cilium.io/)**

> 💡 Efficient eBPF-based networking for containers.

![Cilium](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wvark5crabe1rtv4scbj.png)

Cilium is a free and open-source software solution that provides networking and security 🛡️ for containerized applications.

Cilium leverages eBPF to provide high-performance networking and security and has been designed to integrate with container orchestration platforms like **Kubernetes**. ☸️

One important feature of Cilium is its ability to apply security rules at the network level, also allows us to encrypt and authenticate network traffic, making sure that data is transmitted securely between containers. 🐋

> In short, Cilium secures container workloads in **Kubernetes** with eBPF, offering network security, encryption, and authentication.

To better understand how Cilium works at its architectural level, take a look at the below image. 👇🏻

![Cilium Architecture](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/orj4pimzs73lb3kqcoxs.png)

{% cta https://github.com/cilium/cilium
 %} 🌟 Cilium on GitHub {% endcta %}

***

### 6. **[Rook](https://https://rook.io/)**

> 💡 Simplifies cloud-native persistent storage management.

![Rook.io persistent storage management](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/isvy9wcqkgeziptxmzva.png)

> This tool really is a lifesaver for anyone who is working with the Ceph storage. 😍

To understand and implement Rook, you must have a basic understanding of **Ceph** software-defined storage. If not, let me briefly introduce Ceph.

> What exactly is Ceph? 🧐

Ceph is an open-source software storage platform that provides object storage, block storage, and file system storage in a single unified storage cluster. 📦

Rook simplifies the deployment and management of Ceph storage systems on Kubernetes. 

By deploying Rook on Kubernetes, we can make Ceph storage highly available, automated 🤖, and simple to operate at scale.

> In short, Rook turns Ceph into a fully managed service for Kubernetes and abstracts away the complexity of Ceph, making it easy to set up and manage cloud-native storage for our containers.

{% cta https://github.com/rook/rook
 %} 🌟 Rook on GitHub {% endcta %}

***

### **Introducing [**Odigos**](https://github.com/keyval-dev/odigos)**

> 💡Distributed tracing without code changes.

![keyval-dev banner](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2qeln9ptv8ftdrxc086c.jpeg)

If you are working with OpenTelemetry then this is the tool you need. ✨

Odigos automatically generates telemetry data in OpenTelemetry format to any Observability backend, without any code change. 😲

It achieves this all by the following:

- **Automated Instrumentation 🤖:** Odigos enables automated instrumentation of applications with OpenTelemetry and eBPF, eliminating the need for code modifications.

<br />

- **Universal Observability Tool Compatibility 🤝:** It smoothly integrates with various observability tools, providing comprehensive support and efficient collector management.

<br />

- **Ease of installation ✨:** Odigos excels in user-friendliness, seamlessly integrating OpenTelemetry, ensuring easy installation and adaptability to specific requirements with minimal hurdles.

> We've recently published an article on implementing Odigos. Read [here](https://dev.to/odigos/how-to-monitor-your-requests-between-multiple-applications-4fi8).

<hr />

If you think of any other helpful projects that I haven't covered in this article, please share them in the comments section below. 👇🏻

So, that is it for this article.

Thank you so much for reading! 🎉🫡