Load Balancing with Nginx (Bonus: OpenReplay Introduction)
Published: 2024-01-08 09:28:36

---
title: Load Balancing with Nginx (Bonus: OpenReplay Introduction)
layout: post
description: We identified three solutions to tackle this challenge: 1. GitLab Feature Flags, 2. Host Provider Weighted Load Balance, 3. Load Balancing with Nginx...
---

## What is the problem?

After discovering renowned debugging tools such as **Sentry** and **Open Replay**, I made the decision to integrate them into one of our end-user applications with high traffic. However, the challenge arose in terms of performance and bundle size. With Sentry contributing more to the increase and Open Replay less, our bundle size surged by up to 30%. That's not ideal, is it?

To address this, we initiated a second production pipeline named `Main Debug`, directing only 5% of users to it. This proved sufficient for our debugging needs, and thus the journey begins from this point.

## Solutions

We identified three solutions to tackle this challenge:

### 1. [GitLab Feature Flags](#gitlab-feature-flags)
### 2. [Host Provider Weighted Load Balance](#host-provider-weighted-load-balance)
### 3. [Load Balancing with Nginx](#load-balancing-with-nginx)

---

## GitLab Feature Flags

![Route: Deploy > Feature flags](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9i6p76ucluwfessjibrx.png "Route: Deploy > Feature flags")
<figcaption>Route: Deploy > Feature flags</figcaption>

As A/B testing gained popularity and captured everyone's attention, GitLab introduced a feature called "Feature Flags." This feature provides the capability to establish rules for user percentages, allowing for more controlled and strategic testing.

![Clicking on “new feature flag” button](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/idv51o2oksve4qo770xi.png "Clicking on “new feature flag” button")
<figcaption>Clicking on “new feature flag” button</figcaption>

In summary, everything we desired was within reach, and we successfully navigated our way to it. However, a challenge emerged as we needed two separate branches for `main` and `main-debug` to introduce the configuration in `main-debug` while leaving `main` unchanged. Otherwise, although the performance was satisfactory, the bundle size would persist (unless utilizing **Tree Shaking**, a topic I'll delve into in another article). This solution may not be the perfect fit for our case, but feel free to explore its applicability in your scenario. (We'll explore feature flags in more detail in a future article.)

## Host Provider Weighted Load Balance

We also considered another straightforward solution: utilizing **Host Provider's DNS distribution** via `A` records with the IPs of our production-ready pods.

However, we encountered challenges with this approach. There was a potential risk for the IP addresses to change for any reason, and this posed a significant risk. While we could ensure static IPs through coordination with the host provider, another concern arose. Placing the configuration outside our repository meant that certain individuals had access to it, prompting us to discard this solution. Nonetheless, for smaller-scale projects, I believe this is a swift and efficient solution.

## Load Balancing with Nginx

To start, we need to configure our upstream and create only two upstreams—one for `main` and another for `main-debug` pods. Our upstream configuration would resemble the following:

```nginx
upstream app {
    server main-pod-address.svc:3004 weight=95;
    server main-debug-pod-address.svc:3004 weight=5;
}
```

As you can observe, we utilized the `weight` keyword to distribute users between the two branches: 95% and 5%. Feel free to adjust this based on your specific requirements.

However, another issue arose. The default load balancing strategy, **Round Robin**, wasn't suitable for our front-end needs. Our domain received more than just HTML document requests – various chunks and CSS files needed to be retrieved. Using Round Robin could potentially crash the front-end since it might fetch chunks and CSS from the `main-debug` branch, which contains different code. To mitigate this, we opted for a **Custom IP Hash** strategy. (For additional strategies, consult the [HTTP Load Balancing | NGINX Docs](https://docs.nginx.com/nginx/admin-guide/load-balancer/http-load-balancer/)). This resulted in our upstream configuration appearing as follows:

```nginx
upstream app {
    hash $http_ar_real_ip consistent; # we used http_ar_real_ip header for our ips
    server main-pod-address.svc:3004 weight=95;
    server main-debug-pod-address.svc:3004 weight=5;
}
```

After configuring upstream we will step into config our [server block](https://nginx.org/en/docs/beginners_guide.html) in Nginx and our final server block is this:

```nginx
server{
    listen 80;
    listen [::]:80; # For listening to ipv6
    server_name  behtarino.com;
    access_log /dev/stdout;

    location / {
        proxy_pass http://app; # "app" name came from the upstream

        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}
```

And that’s it! now you have balanced users with your desired weight.

---

## Bonus: What is Open Replay?

![OpenReplay showing many users](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/r4tb9sbpcr4y9ckl3cx0.png "OpenReplay showing many users")

Explore User Interactions with OpenReplay, Your Open-Source Session Replay Solution

Discover the power of OpenReplay, an open-source session replay suite designed to empower developers to understand user interactions on their web applications. This invaluable tool allows developers to visually trace and analyze user actions, significantly expediting the troubleshooting process.

![OpenReplay showing one user flow](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/i0fexkd5nup8xgcbwykz.png "OpenReplay showing one user flow")


Beyond simple user action replay, OpenReplay provides comprehensive insights into the inner workings of your website or application. It captures crucial elements such as network activity, console logs, JavaScript errors, store actions/state, page speed metrics, CPU/memory usage, and more.

One standout feature of OpenReplay is its minimal footprint, utilizing a lightweight ~26KB tracker that asynchronously transmits essential data, ensuring minimal impact on overall performance.

For those who prioritize data control and security, OpenReplay is the ideal solution. As a self-hosted tool, all captured data remains within your cloud environment, affording you complete control and ownership. OpenReplay also boasts fine-grained security features for sanitizing user data.

Flexibility is at the core of OpenReplay's deployment strategy, allowing you to deploy it anywhere. It seamlessly integrates with major public cloud providers, including AWS, GCP, Azure, and Digital Ocean.


![OpenReplay Dashboard](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/m7bprlxi37fv8ejviudc.png "OpenReplay Dashboard")

In addition to its session replay capabilities, OpenReplay offers a suite of features, including developer tools, product analytics, and co-browsing. Explore the possibilities with OpenReplay and elevate your understanding of user interactions on your web applications.

## Results

It appears that our journey has reached its conclusion. Yet, I believe there are many more aspects to explore in this ever-evolving landscape. Please contribute and enrich this article further with your insights and experiences.

Thanks for reading. 🌼