ES Index — S3 Snapshot & Restoration:
Tags: snapshot, restore, backup, index
Published: 2020-03-18 17:38:03

---
title: ES Index — S3 Snapshot & Restoration:
published: true
date: 2017-12-15 16:26:00 UTC
tags: snapshot,restore,backup,index
canonical_url: 
---

### ES Index — S3 Snapshot & Restoration:

![](https://cdn-images-1.medium.com/max/484/0*tB85jys_DWuTYI7o.jpg)

The question is... What brings you here? Fed up with all the searches on how to back-up and restore specific indices?

Fear not, for your search quest ends here.!

After going through dozens of tiny gists and manual pages, here it is.. We’ve done all the heavy lifting for you.

The following tutorial was tested on **_elasticsearch V5.4.0_**

And before we proceed, remember:

**Do’s:**

Make sure that the **_elasticsearch version of the backed-up_**

**_cluster’s version <=(lesser than or equals) Restoring Cluster’s version._**

**Dont’s:**

Unless it’s highly necessary;

**_curl -XDELETE ‘http://localhost:9200/nameOfTheIndex_**

**_#deletes a specific index_**

Especially not, when you are drunk!:

**_curl -XDELETE ‘http://localhost:9200/_all_**

**_#deletes all indexes_**

(This is where the drunk part comes in..!!)

![](https://cdn-images-1.medium.com/max/500/0*PaiwTUgOnjUipmrX.jpg)

### Step1: Install S3 plugin Support

```sh
sudo bin/elasticsearch-plugin install repository-s3  
# (or)  
sudo /usr/share/elasticsearch/bin/elasticsearch-plugin install repository-s3
```

It depends on where your elasticsearch-plugin executable is installed. This enables the elasticsearch instance to communicate with the AWS S3 buckets.

### Step2: Input the Snapshot registration settings

**METHOD** : PUT

**URL:** [http://localhost:9200/_snapshot/logs_backup?verify=false&pretty](http://localhost:9200/_snapshot/logs_backup?verify=false&pretty)

**PAYLOAD:**
```json
 {  
 “type”: “s3”,  
 “settings”: {  
 “bucket”: “WWWWWW”,  
 “region”: “us-east-1”,  
 “access_key”: “XXXXXX”,  
 “secret_key”: “YYYYYY”  
 }  
 }
```

**In the URL:**  
- **logs_backup**: Name of the snapshot file

**In the payload JSON:**  
- **bucket**: “WWWWW” is where you enter the name of the bucket.  
- **access_key & secret_key**: The values “XXXXXX” and “YYYYYY” is where we key in the access key and secret key for the buckets based on the _IAM_ policies. If you need any help to find it, here’s a link that should guide you through ([https://aws.amazon.com/blogs/security/wheres-my-secret-access-key/](https://aws.amazon.com/blogs/security/wheres-my-secret-access-key/)).  
- **region**: region where the bucket is hosted (choose any from: [http://docs.aws.amazon.com/general/latest/gr/rande.html](http://docs.aws.amazon.com/general/latest/gr/rande.html)).

This should give a response as ‘**{“acknowledged”: “true”}**’.

### Step3: Cloud-Sync — list all Snapshots

**URL:** [http://localhost:9200/_cat/snapshots/logs_backup?v](http://localhost:9200/_cat/snapshots/logs_backup?v)

**In the URL**:
- **logs_backup**: Name of the snapshot file  
 Time to sync up all the list of snapshots. If all our settings have been synced up just fine; we should end up with a list of indices, close to that of what is shown below:

![](https://cdn-images-1.medium.com/max/811/0*F29ShkFj4tm_F5VC.png)

### Step4: Creating a Snapshot

**METHOD** : PUT

**URL:** [http://localhost:9200/_snapshot/logs_backup/type_of_the_backup?wait_for_completion=true](http://localhost:9200/_snapshot/logs_backup/type_of_the_backup?wait_for_completion=true)

**PAYLOAD:**
```json 
 {  
 “indices”: “logstash-2017.11.21”,  
 “include_global_state”: false,  
 “compress”: true,  
 “encrypt”: true  
 }
```

**In the URL**:  
- **logs_backup**: Name of the snapshot file

- **type_of_the_backup**: Could be any string

**In the payload JSON**:  
- **indices**: Correspond to the index which is to be backed-up to S3 bucket. In the case of multiple indices to back up under a single restoration point, the indices can be entered in the form of an array.  
- **include_global_state**: set to ‘false’ just to make sure there’s cross-version compatibility. **_WARNING If set to ‘true’, the index can be restored only to the ES of the source version._**   
- **compress**: enables compression of the index meta files backed up to S3.  
- **encrypt**: In case if extra encryption on the indices is necessary.

This should give a response as ‘**{“acknowledged”: “true”}**’

### Step5: Restoring a Snapshot:

**METHOD** : PUT

**URL:** [http://localhost:9200/_snapshot/name_of_the_backup/index_to_be_restored/_restore](http://localhost:9200/_snapshot/name_of_the_backup/index_to_be_restored/_restore)

**PAYLOAD:**
```json
 {  
 “ignore_unavailable”: true,  
 “include_global_state”: false  
 }
```

In the URL:  
**logs_backup**  : Name of the snapshot file

**index_to_be_restored**: Any of the index from the id listed in Step:3

In the payload JSON:  
- **ignore_unavailable**: It’s safe to set this to true, to avoid unwanted checks.  
- **include_global_state**: set to ‘false’ just to make sure there’s cross-version compatibility. **WARNING**  **If set to ‘true’, the index can be restored only to the ES of the source version.**

This should give a response as ‘ **{“acknowledged”: “true”}** ’

Et Voila! The restoration is complete.

And Don’t forget to recycle the space corresponding to the index by safely deleting it — Reuse, Reduce & Recycle :)

Happy Wrangling!!!

_Originally published at_ [_https://www.datawrangler.in_](https://www.datawrangler.in/2017/12/es-index-s3-snapshot-restoration.html) _on December 15, 2017._

* * *