Setting up a Cross-Region Replication rule for S3 buckets through AWS CLI.
Tags: aws, s3
Published: 2022-08-06 13:06:00

S3 is an object-storage service provided by AWS. Object storage takes each piece of data and designates it as an object. Data is kept in separate storehouses versus files in folders and is bundled with associated metadata and a unique identifier to form a storage pool.

This S3 object-based storage is accessible via a Web API which makes it easy to use, against a block-storage type like EBS which has to be attached to an EC2 Instance in order to access it.

You don't have to pay for creating a S3 bucket but you have pay for the Objects! That means every file uploaded to the bucket as well as for how long decides your bill.

**FAQ**
1. What is a Replication Rule?
A replication rule is a rule to replicate objects from a bucket to another.
The rule is attached to a particular bucket (aka Source bucket) which has different parameters, but most importantly the Destination bucket.
The rule automatically replicates the objects to the destination bucket whenever we upload something to the source bucket.

2. What do we mean by cross-region?
Cross-region here means that our buckets are deployed in different regions. For e.g. I deployed a bucket in **us-west-1** (N. California) but I want to the replicate the objects to another bucket deployed in **ap-northeast-1** (Tokyo)

3. Why do we need a Replication between the buckets?
Mainly for Disaster Recovery (DR). Whenever the bucket hosted in our primary datacenter goes down, we can immediately and easily reroute to our secondary datacenter which can be in the same zone or some other zone/region.

So let's start setting up our Replication rule on a bucket!

But before using the CLI, we have to configure it. For the configuration we have to create a **access key** in our AWS acc.
`aws configure` is the command.
![aws configure](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/p31dzw6wcdmmphf4gxmw.png)
Here we gave the access key id and the secret access key obtained from the 'access key' we created.
I took **us-west-1** as my main region and the output format is **json** which is the default one.

## Creating Source and Destination buckets in the CLI

- Let's create a source bucket and enable versioning in it.
![source-bucket](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/a3sw9uoqyqxj7wwp6lid.png)

Create bucket command -> `aws s3api create-bucket --bucket <source-bucket-name> --region <region-code> --create-bucket-configuration LocationConstraint=<region-code>`

Note: When deploying a S3 bucket in **us-east-1** region, we don't have to give the `LocationConstraint` tag.

Enable Versioning command -> `aws s3api put-bucket-versioning --bucket <source-bucket-name> --versioning-configuration Status=Enabled`

- Now let's create a destination bucket in another region and enable versioning in it.
![destination-bucket](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/co05pyyuiugn1mizhci2.png)

Create bucket command -> `aws s3api create-bucket --bucket <destination-bucket-name> --region us-east-1`

Note: Since we deployed this destination bucket in the **us-east-1** region, you can see that we don't have to use the LocationConstraint tag.

Enable Versioning command -> `aws s3api put-bucket-versioning --bucket <destination-bucket-name> --versioning-configuration Status=Enabled`

Ok, we can see that our source and destination buckets are deployed in different regions...
![bucket-deployements](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5s6h8yje6nfsoziauubm.png)

------------------------------------------------------------------

## Creating an IAM role and Replication Rule
In order to create a Replication rule, we need an IAM role to attach the rule to the source bucket. For creating an IAM role, we need 2 JSON files. First for the trust-policy and the Second for the role-permissions.

```
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "s3.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}
```
Save this code as **s3-trust-policy.json**

Then use the following command to create an IAM role and attach the trust-policy to it,
`aws iam create-role --role-name S3replicationRole --assume-role-policy-document file://s3-trust-policy.json`

![s3ReplicationIAMRole](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/a661le31xk1yz4kp7x38.png)

Note: Save the json files in the directory where you've launched your CMD else you have to properly give the file location.

Now attach a Role Permission to the IAM role,

```
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObjectVersionForReplication",
        "s3:GetObjectVersionAcl",
        "s3:GetObjectVersionTagging"
      ],
      "Resource": ["arn:aws:s3:::<source-bucket-name>/*"] 
    },
    {
      "Effect": "Allow",
      "Action": ["s3:ListBucket", "s3:GetReplicationConfiguration"],
      "Resource": ["arn:aws:s3:::source-bucket"]
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:ReplicateObject",
        "s3:ReplicateDelete",
        "s3:ReplicateTags"
      ],
      "Resource": "arn:aws:s3:::<destination-bucket-name>/*" 
    }
  ]
}
```
Replace <source-bucket-name> and <destination-bucket-name> with your respective source and destination bucket names and save it as **s3-role-perms.json**

Now attach the file to the role with the following command,
`aws iam put-role-policy --role-name S3replicationRole --policy-document file://s3-role-perms.json --policy-name S3replicationRolePolicy`

![S3replicationRolePolicy](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/65nchrolr68u4iuagk89.png)

------------------------------------------------------------------

Now let's create a Replication Config file which will set the Replication Rule to our Source bucket.

```
{
  "Role": "<Role-ARN>",
  "Rules": [
    {
      "Status": "Enabled",
      "Priority": 1,
      "DeleteMarkerReplication": { "Status": "Disabled" },
      "Filter": { "Prefix": "Documents" },
      "Destination": {
        "Bucket": "arn:aws:s3:::<destination-bucket-name>"
      }
    }
  ]
}
```
Replace <Role-ARN> with the ARN (Amazon Resource Number) of the IAM Role we created in the previous steps ...
![Role-ARN](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/d8hbfmaa8m1b6odhwngp.png)

... and <destination-bucket-name> with the name of the Destination bucket & save the file as **replicationConf.json**

Then with the help of the following command, attach this config file to the source bucket to set up the Replication Rule,
`aws s3api put-bucket-replication --replication-configuration file://replicationConf.json --bucket <source-bucket-name>`

![Replication Rule](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xrimj2qp17ng1zhpxtw4.png)

------------------------------------------------------------------

Let us verify whether the Replication Rule is properly attached or not!

Use the command to check in the CLI,
`aws s3api get-bucket-replication --bucket <source-bucket-name>`

![CLI-verify](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9inw591eee65bvr92szo.png)

or we can check in the console,
![Console-verify](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2bblo8ytrxmtrdsw3nfy.png)