How to secure gRPC connection with SSL/TLS in Go
Tags: go, security, tutorial, webdev
Published: 2020-07-23 08:45:57

In the [previous lecture](https://dev.to/techschoolguru/use-grpc-interceptor-for-authorization-with-jwt-1c5h), we have learned how to use gRPC interceptors to authenticate users.

However, the API that we used to login user was insecure, which means the username and password were being sent in plaintext, and can be read by anyone who listens to the communication between the client and server. 

So today we will learn how to secure the gRPC connection using TLS.

{% youtube jmqLJMFS_yI %}

If you haven’t read [my post about SSL/TLS](https://dev.to/techschoolguru/a-complete-overview-of-ssl-tls-and-its-cryptographic-system-36pd). I highly recommend you to read it first to have a deep understanding about TLS before continue.

Here's the link to the [full gRPC course playlist on Youtube](https://www.youtube.com/playlist?list=PLy_6D98if3UJd5hxWNfAqKMr15HZqFnqf)
Github repository: [pcbook-go](https://github.com/techschool/pcbook-go) and [pcbook-java](https://github.com/techschool/pcbook-java)
Gitlab repository: [pcbook-go](https://gitlab.com/techschool/pcbook) and [pcbook-java](https://gitlab.com/techschool/pcbook-java)

## Types of gRPC connections

There are 3 types of gRPC connections:

- The first one is insecure connection, which we’ve been using since the beginning of this course. In this connection, all data transfered between client and server is not encrypted. So please don’t use it in production!

![grpc-connection-types](https://dev-to-uploads.s3.amazonaws.com/i/mtvrmqjzikbcr8sub6n8.png)

- The second type is connection secured by server-side TLS. In this case, all the data is encrypted, but only the server needs to provide its TLS certificate to the client. You can use this type of connection if the server doesn’t care which client is calling its API.
- The third and strongest type is connection secured by mutual TLS. We use it when the server also needs to verify who’s calling its services. So in this case, both client and server must provide their TLS certificates to the other.

In this article, we will learn to implement both server-side and mutual TLS in Golang. So let’s get started!

## Generate TLS certificates

First we write the `cert/gen.sh` script to generate TLS certificates.

```bash
rm *.pem

# 1. Generate CA's private key and self-signed certificate
openssl req -x509 -newkey rsa:4096 -days 365 -nodes -keyout ca-key.pem -out ca-cert.pem -subj "/C=FR/ST=Occitanie/L=Toulouse/O=Tech School/OU=Education/CN=*.techschool.guru/emailAddress=techschool.guru@gmail.com"

echo "CA's self-signed certificate"
openssl x509 -in ca-cert.pem -noout -text

# 2. Generate web server's private key and certificate signing request (CSR)
openssl req -newkey rsa:4096 -nodes -keyout server-key.pem -out server-req.pem -subj "/C=FR/ST=Ile de France/L=Paris/O=PC Book/OU=Computer/CN=*.pcbook.com/emailAddress=pcbook@gmail.com"

# 3. Use CA's private key to sign web server's CSR and get back the signed certificate
openssl x509 -req -in server-req.pem -days 60 -CA ca-cert.pem -CAkey ca-key.pem -CAcreateserial -out server-cert.pem -extfile server-ext.cnf

echo "Server's signed certificate"
openssl x509 -in server-cert.pem -noout -text
```

I encourage you to read [my post about how to create and sign TLS certificate](https://dev.to/techschoolguru/how-to-create-sign-ssl-tls-certificates-2aai) to understand how this script works.

Basically this script contains 3 parts:

- First, generate CA’s private key and its self-signed certificate.
- Second, create web server’s private key and CSR.
- And third, use CA’s private key to sign the web server’s CSR and get back its certificate.

The generated files that we care about in this video are:

- The CA’s certificate,
- The CA’s private key,
- The server’s certificate,
- And the server’s private key.

I’m gonna add a new command to the Makefile to run the certificate generation script. We just `cd` to the cert folder, run `gen.sh`, then get out of that folder.

```Makefile
...

cert:
    cd cert; ./gen.sh; cd ..

.PHONY: gen clean server client test cert
```

Now let’s try it in the terminal.

```bash
make cert
```

All files are regenerated successfully.

![cert-generated](https://dev-to-uploads.s3.amazonaws.com/i/y9citnjrk5nd52b5nkbi.png)

Next step, I will show you how to secure our gRPC connection with server-side TLS.

## Implement server-side TLS

Let’s open `cmd/server/main.go` file. I will add a function to load TLS credentials. It will returns a `TranportCredentials` object or an `error`.

```go
func loadTLSCredentials() (credentials.TransportCredentials, error) {
    // Load server's certificate and private key
    serverCert, err := tls.LoadX509KeyPair("cert/server-cert.pem", "cert/server-key.pem")
    if err != nil {
        return nil, err
    }

    // Create the credentials and return it
    config := &tls.Config{
        Certificates: []tls.Certificate{serverCert},
        ClientAuth:   tls.NoClientCert,
    }

    return credentials.NewTLS(config), nil
}
```

For sever side TLS, we need to load server’s certificate and private key. So we use `tls.LoadX509KeyPair()` function to load the `server-cert.pem` and `server-key.pem` files from the `cert` folder. If there’s an error, just return it. Else, we will create the transport credentials from them.

We make a `tls.Config` object with the server certificate, and we set the `ClientAuth` field to `NoClientCert` since we’re just using server-side TLS. Finally we call `credentials.NewTLS()` with that config and return it to the caller.

In the `main()` function, we call `loadTLSCredentials()` to get the TLS credential object. If an error occurs, we just write a fatal log. Otherwise, we add the TLS credential to the gRPC server by using the `grpc.Creds()` option.

```go
func main() {
    ...

    tlsCredentials, err := loadTLSCredentials()
    if err != nil {
        log.Fatal("cannot load TLS credentials: ", err)
    }

    grpcServer := grpc.NewServer(
        grpc.Creds(tlsCredentials),
        grpc.UnaryInterceptor(interceptor.Unary()),
        grpc.StreamInterceptor(interceptor.Stream()),
    )

    ...
}
```

And that’s it for the server. Let’s run it in the terminal.

```bash
make server
```

The server is started. Now run the client:

```bash
make client
```

![failed-client](https://dev-to-uploads.s3.amazonaws.com/i/u0yxstq41vtq5rbeir5q.png)

It failed because we haven’t enabled TLS on the client side yet. So let’s do that!

Similar to what we did on the server, I also add a function to load TLS credentials from files. But this time, we only need to load the certificate of the CA who signed the server’s certificate.

The reason is, client needs to verify the authenticity of the certificate it gets from the server to make sure that it’s the right server it wants to talk to.

```go
func loadTLSCredentials() (credentials.TransportCredentials, error) {
    // Load certificate of the CA who signed server's certificate
    pemServerCA, err := ioutil.ReadFile("cert/ca-cert.pem")
    if err != nil {
        return nil, err
    }

    certPool := x509.NewCertPool()
    if !certPool.AppendCertsFromPEM(pemServerCA) {
        return nil, fmt.Errorf("failed to add server CA's certificate")
    }

    // Create the credentials and return it
    config := &tls.Config{
        RootCAs:      certPool,
    }

    return credentials.NewTLS(config), nil
}
```

So here we load the `ca-cert.pem` file, then create a new x509 cert pool, and append the CA’s pem to that pool. Finally we create the credentials and return it. Note that we only need to set the RootCAs field, which contains the trusted CA’s certificate.

Now in the `main()` function, there are 2 connections which are still insecure. We will need to replace them with the secure TLS.

Let’s call `loadTLSCredentials()` to get the credentials object. Then change the `grpc.WithInsecure()` call to `grpc.WithTransportCredentials()`, and pass in the TLS credentials object that we’ve created.

```go
func main() {
    ...

    tlsCredentials, err := loadTLSCredentials()
    if err != nil {
        log.Fatal("cannot load TLS credentials: ", err)
    }

    cc1, err := grpc.Dial(*serverAddress, grpc.WithTransportCredentials(tlsCredentials))
    if err != nil {
        log.Fatal("cannot dial server: ", err)
    }

    ...

    cc2, err := grpc.Dial(
        *serverAddress,
        grpc.WithTransportCredentials(tlsCredentials),
        grpc.WithUnaryInterceptor(interceptor.Unary()),
        grpc.WithStreamInterceptor(interceptor.Stream()),
    )
    if err != nil {
        log.Fatal("cannot dial server: ", err)
    }

    ...
}
```

And we’re done. Let’s try it out!

![succeed-client](https://dev-to-uploads.s3.amazonaws.com/i/ueaqeoagipkf775n9ddd.png)

This time the requests are successfully sent to the server. Perfect!

## Subject Alternative Name (SAN)

There’s 1 thing I want to show you here. Remember that when we develop on localhost, It’s important to add the `IP:0.0.0.0` as an Subject Alternative Name (SAN) extension to the certificate.

```config
subjectAltName=DNS:*.pcbook.com,DNS:*.pcbook.org,IP:0.0.0.0
```

Let’s see what will happen if we remove this from the `server-ext.cnf` config file.

```config
subjectAltName=DNS:*.pcbook.com,DNS:*.pcbook.org
```

Then regenerate the certificates, restart the server, and run the client again.

```bash
make cert
make server
make client
```

![error-san](https://dev-to-uploads.s3.amazonaws.com/i/nflhs8mtepxccjarntgt.png)

As you can see, there’s an error saying that the TLS handshake failed because it cannot validate the certificate for `0.0.0.0`, since the SAN doesn’t contain this IP address.

On production, it will be OK because we use domain names instead.

Alright, so now you know how to enable server-side TLS for your gRPC connection. Let’s learn how to enable mutual TLS!

## Implement mutual TLS

At the moment, the server has already shared its certificate with the client. For mutual TLS, the client also has to share its certificate with the server. So now let’s update this `cert/gen.sh` script to create and sign a certificate for the client.

```bash
...

# 4. Generate client's private key and certificate signing request (CSR)
openssl req -newkey rsa:4096 -nodes -keyout client-key.pem -out client-req.pem -subj "/C=FR/ST=Alsace/L=Strasbourg/O=PC Client/OU=Computer/CN=*.pcclient.com/emailAddress=pcclient@gmail.com"

# 5. Use CA's private key to sign client's CSR and get back the signed certificate
openssl x509 -req -in client-req.pem -days 60 -CA ca-cert.pem -CAkey ca-key.pem -CAcreateserial -out client-cert.pem -extfile client-ext.cnf

echo "Client's signed certificate"
openssl x509 -in client-cert.pem -noout -text
```

Let’s say for this tutorial, we use the same CA to sign both server and client’s certificates. In the real world, we might have multiple clients with different certificates signed by different CAs.

Now let’s regenearte the certificates.

```bash
make cert
```

OK the client’s certificate and private key are ready. To enable mutual TLS, on the server side `cmd/server/main.go`, we should change the `ClientAuth` field from `tls.NoClientCert` to `tls.RequireAndVerifyClientCert`.

```go
func loadTLSCredentials() (credentials.TransportCredentials, error) {
    ...

    // Create the credentials and return it
    config := &tls.Config{
        Certificates: []tls.Certificate{serverCert},
        ClientAuth:   tls.RequireAndVerifyClientCert,
    }

    return credentials.NewTLS(config), nil
}
```

We also need to provide a list of certificates of the trusted CA who signs our clients’ certificates.

In our case, we only have 1 single CA that signs both server’s and client’s certificate so we can simply copy the codes that we’ve written on the client side to load CA’s certificate and create a new certificate pool.

Then just update the variable names and error messages a bit to reflect the facts that this should be the CA who signs client’s certificate.

```go
func loadTLSCredentials() (credentials.TransportCredentials, error) {
    // Load certificate of the CA who signed client's certificate
    pemClientCA, err := ioutil.ReadFile("cert/ca-cert.pem")
    if err != nil {
        return nil, err
    }

    certPool := x509.NewCertPool()
    if !certPool.AppendCertsFromPEM(pemClientCA) {
        return nil, fmt.Errorf("failed to add client CA's certificate")
    }

    // Load server's certificate and private key
    serverCert, err := tls.LoadX509KeyPair("cert/server-cert.pem", "cert/server-key.pem")
    if err != nil {
        return nil, err
    }

    // Create the credentials and return it
    config := &tls.Config{
        Certificates: []tls.Certificate{serverCert},
        ClientAuth:   tls.RequireAndVerifyClientCert,
        ClientCAs:    certPool,
    }

    return credentials.NewTLS(config), nil
}
```

And we’re done with the server. Let’s run it in the terminal.

```bash
make server
```

Now if we connect the current client to this new server, it will fail because the server now also requires client to send its certificate.

![failed-client-cert](https://dev-to-uploads.s3.amazonaws.com/i/bsvi6strdji4h0oddhtq.png)

Let’s go to the client code `cmd/client/main.go` to fix this. I will just copy the code to load certificate on the server side and change the file names to `client-cert.pem` and `client-key.pem`.

Then we have to add the client certificate to the TLS config by setting the `Certificates` field, just like what we did on the server side.

```go
func loadTLSCredentials() (credentials.TransportCredentials, error) {
    // Load certificate of the CA who signed server's certificate
    pemServerCA, err := ioutil.ReadFile("cert/ca-cert.pem")
    if err != nil {
        return nil, err
    }

    certPool := x509.NewCertPool()
    if !certPool.AppendCertsFromPEM(pemServerCA) {
        return nil, fmt.Errorf("failed to add server CA's certificate")
    }

    // Load client's certificate and private key
    clientCert, err := tls.LoadX509KeyPair("cert/client-cert.pem", "cert/client-key.pem")
    if err != nil {
        return nil, err
    }

    // Create the credentials and return it
    config := &tls.Config{
        Certificates: []tls.Certificate{clientCert},
        RootCAs:      certPool,
    }

    return credentials.NewTLS(config), nil
}
```

OK, now if we re-run the client, all the requests will be successful.

![success](https://dev-to-uploads.s3.amazonaws.com/i/ygvr4l5kge7x90a0yyk4.png)

Awesome!

## Private key encryption

One last thing before we finish. As you know, the current client’s and server’s private key that we are using are not encrypted. It’s because we use the `-nodes` option when generating them (in the `cert/gen.sh` file).

```bash
openssl req -newkey rsa:4096 -nodes -keyout server-key.pem -out server-req.pem -subj "/C=FR/ST=Ile de France/L=Paris/O=PC Book/OU=Computer/CN=*.pcbook.com/emailAddress=pcbook@gmail.com"
```

If we remove this `-nodes` option and run `make cert`, we will be asked to provide a passphrase to encrypt the server’s private key:

![passphrase](https://dev-to-uploads.s3.amazonaws.com/i/hmjs7joiyzz0qk82d4dj.png)

And the generated private key of the server is now encrypted. If we try to start the server with this key, it will return an error: cannot load TLS credentials. That’s because the key is encrypted.

![key-encrypted](https://dev-to-uploads.s3.amazonaws.com/i/bvgt72dzdpkzdsq2g5uj.png)

We can add more codes to decrypt the key with the passphrase, but I think in the end, we still have to protect the passphrase by keeping it somewhere safe. So we can always store our unencrypted private key in that place as well.

For example, if you use amazon web service, you can store your private key or any other secrets in encrypted format with aws secret manager. Or you can use HashiCorp’s Vault for the same purpose.

And that’s everything I wanted to share with you in this article. I hope you find it useful. Thanks a lot for reading, and see you guys in the next one!

---

If you like the article, please [subscribe to our Youtube channel](http://bit.ly/techschoolguru) and [follow us on Twitter](https://twitter.com/techschoolguru) for more tutorials in the future.

---
If you want to join me on my current amazing team at Voodoo, check out [our job openings here](https://bit.ly/voodoojobs). Remote or onsite in Paris/Amsterdam/London/Berlin/Barcelona with visa sponsorship.