Load balancing gRPC service with Nginx
Tags: go, tutorial, webdev, devops
Published: 2020-10-11 20:51:19

So far we’ve learned a lot about how to develop backend web services with gRPC. When it comes to deployment, one important thing we should consider is load balancing.

A large scale gRPC deployment typically has a number of identical backend servers and a number of clients. Load balancing is used for distributing the load from clients optimally across available servers.

{% youtube H93Ric_DE-w %}

Here's the link to the [full gRPC course playlist on Youtube](https://www.youtube.com/playlist?list=PLy_6D98if3UJd5hxWNfAqKMr15HZqFnqf)
Gitlab repository: [pcbook-go](https://gitlab.com/techschool/pcbook) and [pcbook-java](https://gitlab.com/techschool/pcbook-java)

## Table of contents

- [Types of load balancing](#types-of-load-balancing)
  - [Server-side load balancing](#server-side-load-balancing)
  - [Client-side load balancing](#client-side-load-balancing)
  - [Pros and cons](#pros-and-cons)
- [Code refactoring](#code-refactoring)
  - [Update server](#update-server)
  - [Update client](#update-client)
  - [Test the new flag](#test-the-new-flag)
- [Update Makefile](#update-makefile)
- [Install Nginx](#install-nginx)
- [Config Nginx for insecure gRPC](#config-nginx-for-insecure-grpc)
- [Config Nginx for gRPC with TLS](#config-nginx-for-grpc-with-tls)
  - [Enable TLS on Nginx but keep gRPC servers insecure](#enable-tls-on-nginx-but-keep-grpc-servers-insecure)
  - [Enable TLS on both Nginx and gRPC servers](#enable-tls-on-both-nginx-and-grpc-servers)
  - [Multiple routing locations](#multiple-routing-locations)

## Types of load balancing

There are 2 main options for gRPC load balancing: server-side and client-side. Deciding which one to use is a primary architectural choice.

### Server side load balancing

In server-side load balancing, the client issues RPCs to a load balancer or proxy, such as `Nginx` or `Envoy`. The load balancer distributes the RPC call to one of the available backend servers.

It also keeps track of load on each server and implements algorithms for distributing load fairly. The clients themselves do not know about the backend servers.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/wrjdr82cbnjcjcf71fop.png)

### Client side load balancing

In client-side load balancing, the client is aware of multiple backend servers and chooses one to use for each RPC. Usually, the backend servers register themselves with a service discovery infrastructure, such as `Consul` or `Etcd`. Then the client communicates with that infrastructure to know the addresses of the servers.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/8uw4tujh6wvsj4sehkws.png)

A thick client implements the load balancing algorithms itself. For example, in a simple configuration, where the server load is not considered, the client can just round-robin between available servers.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/ox8lr98vj8aod4fr30bi.png)

Another approach is to use a look-aside load balancer, where the load balancing smarts are implemented in a special load-balancing server. Clients query the look-aside load balancer to get the best server(s) to use. The heavy lifting of keeping server state, service discovery, and implementation of a load balancing algorithm is consolidated in the look-aside load balancer.

### Pros and cons

One pros of server-side load balancing is simple client implementation. All client needs to know is the address of the proxy, no more coding is needed. This approach works even for untrusted clients, which means the gRPC service can be open for everyone from the public internet.

However, its cons is adding 1 more extra hop to the call. All RPCs have to go through the proxy before reaching the backend server, thus causing higher latency. Therefore, this server-side load balancing is suitable for the cases where there are many clients, possibly untrusted from the open internet who want to connect to our gRPC servers in a data center.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/bu7rzx7a76mp56v61nw8.png)

The client-side load balancing, on the other hand, doesn’t add any extra hop to the call and thus giving us higher performance in general. However, the client implementation now becomes complex, especially for thick client approach. Therefore, it should only be used for trusted clients, or we will need to use a look-aside load balancer to stand in the front of the trust boundary network. Client-side load balancing is often used in a very high traffic system and microservices architecture.

In this article, we will learn how to set up server-side load balancing for our `gRPC` services with `Nginx`.

## Code refactoring

Since I’m gonna show you different `Nginx` configurations where TLS can be enabled or disabled on the server and client, let’s update our code a bit to take in a new command-line argument.

### Update server

On the server, let's add a new boolean flag `enableTLS`, which will tell us whether we want to enable TLS on our gRPC server or not. Its default value is `false`.

```go
func main() {
    port := flag.Int("port", 0, "the server port")
    enableTLS := flag.Bool("tls", false, "enable SSL/TLS")

    flag.Parse()
    log.Printf("start server on port %d, TLS = %t", *port, *enableTLS)

    ...
}
```

Then let's extract the interceptors to a separate `serverOptions` variable. We check the `enableTLS` flag. Only in case it’s enabled then we load the TLS credentials, and append that credentials to the server options slice. Finally we just pass the server options to the `grpc.NewServer()` function call.

```go
func main() {
    ...

    interceptor := service.NewAuthInterceptor(jwtManager, accessibleRoles())
    serverOptions := []grpc.ServerOption{
        grpc.UnaryInterceptor(interceptor.Unary()),
        grpc.StreamInterceptor(interceptor.Stream()),
    }

    if *enableTLS {
        tlsCredentials, err := loadTLSCredentials()
        if err != nil {
            log.Fatal("cannot load TLS credentials: ", err)
        }

        serverOptions = append(serverOptions, grpc.Creds(tlsCredentials))
    }

    grpcServer := grpc.NewServer(serverOptions...)

    ...
}
```

And that’s it for the server. Let’s do similar thing for the client!

### Update client

First we add the `enableTLS` flag to the command line argument. Then we define a `transportOption` variable  with the default value `grpc.WithInsecure()`.

```go
func main() {
    serverAddress := flag.String("address", "", "the server address")
    enableTLS := flag.Bool("tls", false, "enable SSL/TLS")

    flag.Parse()
    log.Printf("dial server %s, TLS = %t", *serverAddress, *enableTLS)

    transportOption := grpc.WithInsecure()

    if *enableTLS {
        tlsCredentials, err := loadTLSCredentials()
        if err != nil {
            log.Fatal("cannot load TLS credentials: ", err)
        }

        transportOption = grpc.WithTransportCredentials(tlsCredentials)
    }

    cc1, err := grpc.Dial(*serverAddress, transportOption)
    if err != nil {
        log.Fatal("cannot dial server: ", err)
    }

    authClient := client.NewAuthClient(cc1, username, password)
    interceptor, err := client.NewAuthInterceptor(authClient, authMethods(), refreshDuration)
    if err != nil {
        log.Fatal("cannot create auth interceptor: ", err)
    }

    cc2, err := grpc.Dial(
        *serverAddress,
        transportOption,
        grpc.WithUnaryInterceptor(interceptor.Unary()),
        grpc.WithStreamInterceptor(interceptor.Stream()),
    )
    if err != nil {
        log.Fatal("cannot dial server: ", err)
    }

    laptopClient := client.NewLaptopClient(cc2)
    testRateLaptop(laptopClient)
}
```

Only when the `enableTLS` flag value is `true`, we load the TLS credentials from PEM files and change the `transportOption` to `grpc.WithTransportCredentials(tlsCredentials)`. Finally we pass the `transportOption` to the grpc connections. And the client is done.

### Test the new flag

Now if we run `make server`, we can see that the server is running with TLS disabled.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/jsvi6knk47y6hfv2o77j.png)

And if we run make client, it’s also running with no TLS, and all the RPC calls are successful.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/vkcypl9egkj61q2bd029.png)

If we add `-tls` flag to the `make server` command, and restart it, the TLS will be enabled.

```Makefile
...

server:
    go run cmd/server/main.go -port 8080 -tls

...
```

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/ecaoh6gh5k4snen6tv00.png)

If we run `make client` now, the requests will fail:

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/htqmlv12w16qtm5skddr.png)

We have to enable TLS on client side as well, by adding `-tls` flag to the `make client` command.

```Makefile
...

client:
    go run cmd/client/main.go -address 0.0.0.0:8080 -tls

...
```

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/h2ibukx36w8pedexhdh4.png)

And now we can see the requests are successful again.

## Update Makefile

Alright, now the TLS flag is working as we wanted. Before adding `Nginx`, let's update our `Makefile` a bit so that we can easily run multiple instances of the server and the client with or without TLS.

I’m gonna remove the `-tls` flags so that the `make server` and `make client` commands will run without TLS. And I will add 2 more make commands to run 2 instances of the server on different ports. Let’s say the 1st server will run on port `50051`, and the 2nd sever will run on port `50052`.

```Makefile
...

server:
    go run cmd/server/main.go -port 8080

client:
    go run cmd/client/main.go -address 0.0.0.0:8080

server1:
    go run cmd/server/main.go -port 50051

server2:
    go run cmd/server/main.go -port 50052

...
```

Let’s also add 3 more make commands to start the client and servers with TLS. The `client-tls` command will run the client with TLS. The `make server1-tls` command will start a TLS server on port `50051`, and the `make server2-tls` command will start another TLS server on port `50052`.

```Makefile
...

client-tls:
    go run cmd/client/main.go -address 0.0.0.0:8080 -tls

server1-tls:
    go run cmd/server/main.go -port 50051 -tls

server2-tls:
    go run cmd/server/main.go -port 50052 -tls

...
```

## Install Nginx

The next thing we need to do is to install `Nginx`. Since I’m on a mac, I can simply use Homebrew:

```bash
❯ brew install nginx
```

After nginx is installed, we can go to this `usr/local/etc/nginx` folder to config it. Let's open the `nginx.conf` file with with visual studio code.

```bash
❯ cd /usr/local/etc/nginx
❯ code nginx.conf
```

This is the default configuration:

```conf
#user  nobody;
worker_processes  1;

#error_log  logs/error.log;
#error_log  logs/error.log  notice;
#error_log  logs/error.log  info;

#pid        logs/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       mime.types;
    default_type  application/octet-stream;

    #log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
    #                  '$status $body_bytes_sent "$http_referer" '
    #                  '"$http_user_agent" "$http_x_forwarded_for"';

    #access_log  logs/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    #keepalive_timeout  0;
    keepalive_timeout  65;

    #gzip  on;

    server {
        listen       8080;
        server_name  localhost;

        #charset koi8-r;

        #access_log  logs/host.access.log  main;

        location / {
            root   html;
            index  index.html index.htm;
        }

        #error_page  404              /404.html;

        # redirect server error pages to the static page /50x.html
        #
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }

        # proxy the PHP scripts to Apache listening on 127.0.0.1:80
        #
        #location ~ \.php$ {
        #    proxy_pass   http://127.0.0.1;
        #}

        # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
        #
        #location ~ \.php$ {
        #    root           html;
        #    fastcgi_pass   127.0.0.1:9000;
        #    fastcgi_index  index.php;
        #    fastcgi_param  SCRIPT_FILENAME  /scripts$fastcgi_script_name;
        #    include        fastcgi_params;
        #}

        # deny access to .htaccess files, if Apache's document root
        # concurs with nginx's one
        #
        #location ~ /\.ht {
        #    deny  all;
        #}
    }

    include servers/*;
}
```

There are several things that we don’t need to care about in this tutorial, so let's update this config file.

## Config Nginx for insecure gRPC

First, let's remove the user config, uncomment the error log, remove the config for log levels and process id, and let’s say we just need 10 worker connections for now.

One important thing we need to do is to config the right location to store the error log and access log files. In my case, Homebrew has already created a log folder for `Nginx` at `/usr/local/var/log/nginx`, so I just go ahead and use it in the error/access log setting.

```conf
worker_processes  1;

error_log  /usr/local/var/log/nginx/error.log;

events {
    worker_connections  10;
}

http {
    access_log  /usr/local/var/log/nginx/access.log;

    server {
        listen       8080 http2;

        location / {
        }
    }
}
```

Now in the server block, we have a `listen` command to listen to incoming requests from client on port `8080`. This is the default config for a normal HTTP server. Since gRPC uses `HTTP/2`, we should add `http2` at the end of this command.

Let’s remove the server name and charset since we don’t need them now. Similar for the access log because we’ve already defined it above. Let’s also delete the config for default root HTML file and everything after the `location` block as we don’t care about them for now.

OK, now we want to load balance the incoming requests to our 2 server instances. So we should define a `upstream` for them. I’m gonna call it `upstream pcbook_services`. Inside this block, we use the `server` keyword to declare a server instance. The first one is running on `localhost` port `50051`, and the second one is running on port `50052`.

```conf
worker_processes  1;

error_log  /usr/local/var/log/nginx/error.log;

events {
    worker_connections  10;
}

http {
    access_log  /usr/local/var/log/nginx/access.log;

    upstream pcbook_services {
        server 0.0.0.0:50051;
        server 0.0.0.0:50052;
    }

    server {
        listen       8080 http2;

        location / {
            grpc_pass grpc://pcbook_services;
        }
    }
}
```

Then to route all RPC calls to the upstream, in the `location` block,  we use `grpc_pass` keyword, followed by the `grpc://` scheme and the name of the upstream, which is `pcbook_services`.

And that’s it! The load balancing for our insecure gRPC server is done.

Let’s run nginx in the terminal to start it.

```bash
❯ nginx
❯ ps aux | grep nginx
quangpham         9013   0.0  0.0  4408572    800 s000  S+    6:13PM   0:00.00 grep --color=auto --exclude-dir=.bzr --exclude-dir=CVS --exclude-dir=.git --exclude-dir=.hg --exclude-dir=.svn --exclude-dir=.idea --exclude-dir=.tox nginx
quangpham         9007   0.0  0.0  4562704   1124   ??  S     6:12PM   0:00.00 nginx: worker process
quangpham         9006   0.0  0.0  4422416    612   ??  Ss    6:12PM   0:00.00 nginx: master process nginx
```

We can check if it’s running or not using the `ps` and `grep` command. Let’s check out the log folder:

```bash
❯ cd /usr/local/var/log/nginx
❯ ls -l
total 0
-rw-r--r--  1 quangpham  admin  0 Oct 11 18:12 access.log
-rw-r--r--  1 quangpham  admin  0 Oct 11 18:12 error.log
```

As you can see, 2 log files are generated: `access.log` and `error.log`. They’re empty at the moment because we haven’t sent any requests yet.

Now let’s run `make server1` to start the first server on port `50051` with `TLS = false`. Then on another tab, run `make server2` to start the second server on port `50052`, also with TLS disabled. Finally, we run `make client` on another new tab.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/w8bt7snvobbdy2vilrfz.png)

Looks good. All RPC calls are successful. Let’s check the logs on our servers.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/qd295l5kwu9ny8ytmdma.png)

The `server2` receives 2 create laptop requests.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/kjywek4xvh4h2nxyr20p.png)

And the `server1` receives 1 login request and 1 create laptop request. Excellent!

And after a while, there’s another login request comming to this server. That’s because our client is still running, and it periodically calls login to refresh the token.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/gd1ozqq78tnjaselr6hi.png)

I hope you still remember the codes that we’ve written in the [gRPC interceptors lecture](https://dev.to/techschoolguru/use-grpc-interceptor-for-authorization-with-jwt-1c5h).

OK, now let’s look at the nginx access log file.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/qpurkedbib8shko4l3f9.png)

You can see first there’s a login call, then 3 create laptop calls, and finally another login call. So everything is working exactly as we expect.

Next I’m gonna show you how to enable `SSL/TLS` for `Nginx`.

## Config Nginx for gRPC with TLS

In a typical deployment, the gRPC servers are already running inside a trusted network, and only the load balancer (`Nginx` in this case) is exposed to the public internet. So we can leave our gRPC servers running without `TLS` as before, and only add `TLS` to `Nginx`.

### Enable TLS on Nginx but keep gRPC servers insecure

To do that, we will need to copy 3 pem files to the nginx config folder:

- The server’s certificate
- The server’s private key
- And the certificate of the CA who signed client’s certificate if we use mutual TLS.

OK, so now I’m gonna `cd` to the `/usr/local/etc/nginx` folder and create a new `cert` folder. Then I will copy those 3 pem files from our `pcbook` project to this folder.

```bash
❯ cd /usr/local/etc/nginx
❯ mkdir cert
❯ cp ~/Projects/techschool/pcbook-go/cert/server-cert.pem cert
❯ cp ~/Projects/techschool/pcbook-go/cert/server-key.pem cert
❯ cp ~/Projects/techschool/pcbook-go/cert/ca-cert.pem cert
```

Alright, now all the certificate and key files are ready. Let's go back to our nginx config file.

To enable TLS, we first need to add `ssl` to the `listen` command. Then we use the `ssl_certificate` command to give `Nginx` the location of the server’s certificate file. And use the `ssl_certificate_key` command to give it the location of the server’s private key file.

```conf
...

    server {
        listen       8080 ssl http2;

        ssl_certificate cert/server-cert.pem;
        ssl_certificate_key cert/server-key.pem;

        ssl_client_certificate cert/ca-cert.pem;
        ssl_verify_client on;

        location / {
            grpc_pass grpc://pcbook_services;
        }
    }

...
```

As we’re using mutual TLS, we also need to use the `ssl_client_certificate` command to tell nginx the location of the client CA’s certificate file. And finally we set `ssl_verify_client` to `on` to tell nginx to verify the authenticity of the certificate that client will send.

And we’re done. Let’s restart nginx. We run `nginx -s stop` to stop it first. Then we start it with `nginx` command.

```bash
❯ nginx -s stop
❯ nginx
```

Our server is already running, so let’s run the client!

If we just run `make client`, it will run without TLS, so the request will fail, because `Nginx` is now running with TLS enabled.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/qhqhoj3ogvln1pxmb3vp.png)

Now let’s run `make client-tls`.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/pvav47plj3p88apu1bpl.png)

This time the client is running with TLS, and all requests are successful.

Keep in mind that our servers are still running without TLS. So basically what happens is: only the connection between client and `Nginx` is secure, and `Nginx` is connecting to our backend servers via another insecure connection.

Once `Nginx` receives the encrypted data from client, it will decrypt the data before forwarding it to the backend servers. Therefore, you should only use this approach if the `Nginx` and backend servers stay in the same trusted network.

OK, but what if they are not on the same trusted network? Well, in that case, we have no choice but to enable TLS on our backend servers as well, and config nginx to work with it.

### Enable TLS on both Nginx and gRPC servers

Let’s stop the current `server1` and `server2`, then restart them with TLS.

```bash
❯ make server1-tls
❯ make server2-tls
```

Now if we run `make client-tls` immediately, the request will fail.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/pqlnyoquw6gy8wd9xlem.png)

The reason is, although the TLS handshake between the client and `Nginx` succeeded, the TLS handshake between `Nginx` and our backend servers failed since the backend servers are now expecting a secure TLS connection, while `Nginx` is still using an insecure connection when connecting to the backend servers.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/v7p40hhyb4hbec0qwbgu.png)

As you can see in the error log, the failure happened when `Nginx` talked to the upstream servers.

To enable secure TLS connection between nginx and upstream, in the `nginx.conf` file, we have to change the `grpc` scheme to `grpcs`.

```conf
...

    server {
        ...

        location / {
            grpc_pass grpcs://pcbook_services;
        }
    }

...
```

This should be enough if we just use server-side TLS. However, in this case, we’re using mutual TLS, so if we just restart `Nginx` now and rerun `make client-tls`, the request will still fail because `Nginx` is not configured to send its certificate to the upstream servers yet.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/3zf37mw29lnokbzt8i24.png)

We’ve got the `bad certificate` error as you can see in the log.

Let’s see what happens if we go to the server code `cmd/server/main.go` and change the `ClientAuth` field from `tls.RequireAndVerifyClientCert` to `tls.NoClientCert`, which means we will just use server-side TLS.

```go
func loadTLSCredentials() (credentials.TransportCredentials, error) {
    ...

    // Create the credentials and return it
    config := &tls.Config{
        Certificates: []tls.Certificate{serverCert},
        ClientAuth:   tls.NoClientCert,
        ClientCAs:    certPool,
    }

    return credentials.NewTLS(config), nil
}
```

Then restart `server1-tls` and `server2-tls`, and run `make client-tls` again.

```bash
❯ make server1-tls
❯ make server2-tls
❯ make client-tls
```

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/5cu8vcibjqcy9afdu22u.png)

This time all requests are successful. That's exactly what we expected!

OK now what if we really want mutual TLS between nginx and upstream?

Let’s change the `ClientAuth` field back to `tls.RequireAndVerifyClientCert`, restart the 2 TLS backend servers, and get back to our `nginx.conf` file.

This time, we must instruct `Nginx` to do mutual TLS with the backend servers by giving it the location of the certificate and private key. We use the `grpc_ssl_certificate` keyword for the certificate, and the `grpc_ssl_certificate_key` keyword for the private key.

You can generate a different pair of certificate and private key for `Nginx` if you want. Here I simply use the same certificate and private key of the servers.

```conf
worker_processes  1;

error_log  /usr/local/var/log/nginx/error.log;

events {
    worker_connections  10;
}

http {
    access_log  /usr/local/var/log/nginx/access.log;

    upstream pcbook_services {
        server 0.0.0.0:50051;
        server 0.0.0.0:50052;
    }

    server {
        listen       8080 ssl http2;

        # Mutual TLS between gRPC client and nginx
        ssl_certificate cert/server-cert.pem;
        ssl_certificate_key cert/server-key.pem;

        ssl_client_certificate cert/ca-cert.pem;
        ssl_verify_client on;

        location / {
            grpc_pass grpcs://pcbook_services;

            # Mutual TLS between nginx and gRPC server
            grpc_ssl_certificate cert/server-cert.pem;
            grpc_ssl_certificate_key cert/server-key.pem;
        }
    }
}
```

OK, let’s try it.

First stop the current nginx process, then start a new one. And run `make client-tls` again.

```bash
❯ nginx -s stop
❯ nginx
❯ make client-tls
```

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/bmn6mgczshjskc5r09yx.png)

This time all requests are successful. Perfect!

### Multiple routing locations

There’s one more thing I want to show you before we finish.

As you’ve already seen, the login and create-laptop requests are now evenly distributed between our 2 backend servers. But sometimes, we might want to separate the authentication service and the business logic service.

For example, let’s say we want all login requests to go to server 1, and all other requests to go to server 2. In that case, we can also tell `Nginx` to route the requests based on its path.



```conf
worker_processes  1;

error_log  /usr/local/var/log/nginx/error.log;

events {
    worker_connections  10;
}

http {
    access_log  /usr/local/var/log/nginx/access.log;

    upstream auth_services {
        server 0.0.0.0:50051;
        server 0.0.0.0:50052;
    }

    upstream laptop_services {
        server 0.0.0.0:50051;
        server 0.0.0.0:50052;
    }

    server {
        listen       8080 ssl http2;

        # Mutual TLS between gRPC client and nginx
        ssl_certificate cert/server-cert.pem;
        ssl_certificate_key cert/server-key.pem;

        ssl_client_certificate cert/ca-cert.pem;
        ssl_verify_client on;

        location /techschool.pcbook.AuthService {
            grpc_pass grpcs://auth_services;

            # Mutual TLS between nginx and gRPC server
            grpc_ssl_certificate cert/server-cert.pem;
            grpc_ssl_certificate_key cert/server-key.pem;
        }

        location /techschool.pcbook.LaptopService {
            grpc_pass grpcs://laptop_services;

            # Mutual TLS between nginx and gRPC server
            grpc_ssl_certificate cert/server-cert.pem;
            grpc_ssl_certificate_key cert/server-key.pem;
        }
    }
}
```

Here I just copy the path `/techschool.pcbook.AuthService` of the `AuthService` and paste it to this location. Then I change this upstream name to `auth_services`. It should only connect to the `server1` at port `50051`.

Then I add another upstream for the `laptop_services` and make it connect to only `server2` at port `50052`. Then duplicate the location block, change the upstream name to `laptop_services`, and update the path to `techschool.pcbook.LaptopService`.

OK, let’s try this! We just need to restart `Nginx` and run `make client-tls`.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/izwjcjnwb7kmv2u02qop.png)

Now we can see only login request goes to `server1`.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/a9e4mq9k3tvozhvealy7.png)

And all other create laptop requests go to `server2`. Even if we run this make client-tls multiple times.

So it works! And that wraps up our lecture about load balancing gRPC with `Nginx`.

I’m gonna push this nginx config file to the [pcbook-go repository](https://github.com/techschool/pcbook-go) so that you can download and play with it if you like.

Thanks a lot for reading and following the course. Happy coding and see you in the next one!

---

If you like the article, please [subscribe to our Youtube channel](http://bit.ly/techschoolguru) and [follow us on Twitter](https://twitter.com/techschoolguru) for more tutorials in the future.

---
If you want to join me on my current amazing team at Voodoo, check out [our job openings here](https://bit.ly/voodoojobs). Remote or onsite in Paris/Amsterdam/London/Berlin/Barcelona with visa sponsorship.