Use AWS Certificate Manager (ACM) to simplify UI certification management for CockroachDB
Tags: cockroachdb, aws, acm
Published: 2022-11-04 20:45:01

Recently a customer asked me how to use AWS Certificate Manager(ACM) to manage certifications for self-hosted CockroachDB clusters. I looked into it and would like to share tips and tricks below. Please feel free to comment below to let me know know what other topics you like to see!

## Problem statement:

CockroachDB secure cluster requires TLS/HTTPS to access DBConsole, and we currently recommend using Let’s Encrypt and [upload UI Certs to cockroach nodes](https://www.cockroachlabs.com/docs/v22.1/create-security-certificates-custom-ca.html#accessing-the-db-console-for-a-secure-cluster). When certificates expires, this requires additional administrative effort to rotate and maintain the certs.

## Solution:

AWS ACM can issue and auto renew certificates when using DNS validation and thus can reduce administrative overhead. AWS NLB TLS termination can handle TLS decryption between browser and NLB, and re-encryption between NLB and CockroachDB nodes to meet the requirement of CockroachDB. AWS NLB document isn’t very clear on how to configure it, so we’re adding additional screenshots here. 

The other benefit is we now only need to manage Node Certs on the server side.

## Walkthrough:


1. Request a public certificate in ACM for a domain name. I used `jessielin.xxxx.dev` in this case.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/i02ems0yu9kc1w4gbq4l.png)

2. Create Network Load Balancer with 2 listeners. One for DBConsole, and one for SQL access.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/oa7jnm8ldjdoncu1oi0h.png)

a. For DBConsole access, add TLS listener and Target Group. Use the certs issued by ACM. Target Group port should be the port number specified in --http-addr.  By using TLS listener and TLS target group AWS NLB will decrypt and re-encrypt.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/olz9ccuotjp6tgdiiccp.png)
Target Group

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/za5eapqd6hh29oszhr7x.png)
Healthcheck

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/d8brxcik4fm8mj71tr4j.png)

b. For SQL access, add TCP listener and forward it to sql port. The [official document](https://www.cockroachlabs.com/docs/v22.1/deploy-cockroachdb-on-aws.html#step-4-set-up-load-balancing ) explains it very well. 

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4jpbzq3od9grpqkja9b7.png)


3. Add LB hostname and ip address to Node certs 

  a. use `openssl x509 -in certs/node.crt -text` to find out existing nodes' hostname, ip addresses

   b. LB hostname is required to Common Name or Subject Alternative Names fields of the certificate, documented [here](https://www.cockroachlabs.com/docs/stable/authentication.html#using-digital-certificates-with-cockroachdb). To Add LB hostname and ip addr to the list, use `cockroach cert create-node` to create new certs as [documented]( https://www.cockroachlabs.com/docs/v22.1/cockroach-cert.html#create-the-certificate-and-key-pairs-for-nodes) and redistribute to all nodes. Validate the new certs are loaded correctly from DBConsole

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/x63apbwantajy90gnmrx.png)


4. Create an A record in Route 53 to redirect the `jessielin.xxxx.dev` to the NLB

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jqcwjyqo6mv7kvpzfzlr.png)


5. Voila there you have it!

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4gli4r391zk8easwmigz.png)
![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/m2sunkgkinp4ivl5omj7.png)