Automatic DB backup to S3 using AWS CLI
Published: 2022-07-16 17:15:03

**Automatic DB backup to S3 using AWS CLI
**


AWS CLI (Amazon Web Service Command Line Interface) is an open-source command-line utility tool for managing Amazon web services. AWS CLI is a utility tool provided by AWS to manage resources. AWS API is directly accessible through AWS CLI. Using AWS CLI utility, you can create scripts for automating Amazon Web Services. 
AWS CLI is available in version 1 and version 2. AWS recommends using AWS CLI version 2. If you already have version 1, suggest uninstalling it or create a symlink/alias with a different name.
In this tutorial, you will learn how to install AWS CLI on Ubuntu 16.0.7 & 20.04



	
**Installing AWS CLI using APT**
AWS CLI utility tool is available in the default repository of Ubuntu 20.04. To install the tool using APT, follow the following steps:
Update system packages:
Run the following command update system packages and repository index to latest.
# sudo apt-get update
Once the system packages are updated, run the following command to install AWS CLI. 
# sudo apt-get install awscli -y
On completion of the installation, run the following command to verify.
# aws --version
You will get output similar as:


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/b5ywwqahdbfgtjx7790r.png)

After installation, you can configure access to AWS accounts using AWS CLI. Before starting with aws configuration, you should have IAM user's Access key and Secret key provided with proper access. 
Run the following command to configure access to aws account.
# aws configure

Enter the following details accordingly:
AWS Access Key ID [IAM user's Access key]
AWS Secret Access Key [IAM user's secret key]
Default region name [Aws region]
Default output format [JSON format is fine]
You will have output similar as:


**Creating S3 Bucket using AWS CLI**


You can create as S3 bucket using below command followed by aws. 
Now run the following command to create a simple bucket. In this example, I have selected the ap-south-1 region and provided the bucket name 'labuser1-doc'. You can choose your preferred region and bucket name.
# aws s3 mb s3://labuser1-doc --region ap-south-1

You will get output as :


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/uqy0g3zlw4x5nb9aw602.png)

Now check the S3 Bucket list as:
aws s3 ls

You will get the list of created S3 Bucket:


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8aaqx7b9xgeragop6dlk.png)

**Push Contents to S3 Bucket**
Once an S3 bucket is created, you are now good to go for file storage. Run the following command to push your contents to an S3 bucket. In this example, I have sent a simple text file to the S3 bucket. You can push other files in similar ways.
#aws s3 cp labuser1.txt s3://labuser1-doc
Where labuser1.txt is the filename and labuser1-doc is the S3 bucket name. Once the upload is successful, you will get output similar as:
output:


You can list the contents of the bucket using the following command.
#aws s3 ls s3://labuser1-doc


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ylojwa3m4o3bq8l3j33a.png)


**Now create script to automatic backup to S3 bucket**

#!/bin/sh


#Backup DB
date=$(date '+%Y-%m-%d-%H%M')
PGPASSWORD="1234567890" pg_dump --host 127.0.0.1 --port 5432 -U postgres --format custom --blobs --verbose --file "/database/labuser1_$date.bak" labuser1_db

#Remove backups older than 10 days
find /database/labuser1_2022-* -mtime +5 -exec rm {} \;

# Sync to amazon.
aws s3 sync /database s3://labuser1-doc


**Let us save & make the script executable: **

Chmod o+x /script_file



**Conclusion**

This article describes an easy way to backup a Postgres database to Amazon S3 using AWS CLI and crontab in the Linux environment. We will also limit the backup storage duration that saves storage as well in both vm & S3 buckets. 

We have an Ubuntu 16 & 20 VM where PG (PostgreSql) is hosted & cronjob to backup the db twice a day & sync to S3 automatically. We will have a script for auto backup, sync to S3, keeping db of last 7 days at VM & auto delete the last backup files in 30 days in S3 for saving storage & cost.