Blue/Green Deployments for RDS: How Fast is a Switchover? 
Tags: mysql, database, aws
Published: 2022-12-13 11:46:23

AWS recently announced the general availability of [RDS Blue/Green Deployments](https://www.infoq.com/news/2022/12/aws-rds-aurora-blue-green/), a new feature for RDS and Aurora to perform blue/green database updates. One of the aspects that caught my eye is how fast a switchover is. 

> In as fast as a minute, you can promote the staging environment to be the new production environment with no data loss.

Let’s give it a try!  How can we check how long a switchover really takes? We will perform the simplest test, using only a MySQL client and a while loop in Bash:

```
while true; do mysql -s -N -h renato-cluster.***.eu-west-1.rds.amazonaws.com -u renato 
-e "select now()"; sleep 1; done

(...)
2022-12-09 07:58:55
2022-12-09 07:58:56
2022-12-09 07:58:57
2022-12-09 07:58:58
(...)
```
We keep asking every second for the current date and time to the RDS server and we will check what happens when we trigger a switchover. We will compare the results with a reboot and a reboot with failover. To perform the test we will use an m6g.large Multi-AZ instance with some production traffic, a cluster where the _Seconds_Behind_Master_ value of a replica hardly goes above a single-digit number. 


## Rebooting an instance

Using the CLI, we can perform a reboot of an RDS instance without forcing a failover of the database server.
```
aws rds reboot-db-instance --db-instance-identifier renato-cluster --no-force-failover

(...)
2022-12-09 08:00:01
2022-12-09 08:00:02
2022-12-09 08:00:03
2022-12-09 08:00:23
2022-12-09 08:00:24
2022-12-09 08:00:25
(...)
```
That is just about **20 seconds** for a reboot of an RDS instance.


## Rebooting with failover

We can now repeat the test forcing a failover:
```
aws rds reboot-db-instance --db-instance-identifier renato-cluster --force-failover

(...)
2022-12-09 08:01:11
2022-12-09 08:01:12
2022-12-09 08:01:13
2022-12-09 08:03:28
2022-12-09 08:03:29
2022-12-09 08:03:30
(...)
```
This time it takes longer, including the time required to switch the CNAME of our database to the new IP address: **2 minutes and 15 seconds**.


## Switching a blue/green deployment

Finally, we can trigger a switchover in our blue/green deployment calling:
```
aws rds switchover-blue-green-deployment --blue-green-deployment-identifier renato-cluster-bg

(...)
2022-12-09 08:06:59
2022-12-09 08:07:00
2022-12-09 08:07:01
2022-12-09 08:08:07
2022-12-09 08:08:08
2022-12-09 08:08:09
(...)
```
The switchover time of the RDS Blue/Green Deployments is just about a minute: 1 minute and 6 seconds. 


## Recap

- Reboot: 20 seconds
- Reboot with failover: 135 seconds
- Switchover: 66 seconds

The switchover of an RDS for MySQL instance is “as fast as a minute” and significantly faster than a reboot with failover of a Multi-AZ instance. Of course, the switchover can take significantly longer according to the write load of your production database. This was only a basic test to validate the minimum downtime for a switchover.

Questions? Comments? [Contact or follow me](https://www.linkedin.com/in/rlosio/).
Originally published on [cloudiamo.com](https://cloudiamo.com/).