Repository: cloud-custodian/cloud-custodian

Issue:
Is there an ability to add a tag to a snapshot that is created from the ec2 'snapshot' action?
We currently create snapshots from each EC2 instance itself via a cron job and add an Expiration tag at the time of creation.  I'd like to use custodian policies instead for the creation to have it more centrally managed.  The snapshot action works great - just looking to add a tag for how long to keep it.  I already have a policy that looks for an 'Expiration' tag and deletes the snapshot when the time is greater than what is in the tag (which also works great).  

Although I'd love to use a robust solution like mentioned in  #226, my more basic workaround would be to have a tag set at the EC2 instance on how often the instance needs to be backed up (hourly, daily, weekly, etc or any combo), and then have a policy that looks for that tag and creates the snapshot at that time/interval with basic rules that I define on retention based upon that policy.  Although it could be more advanced/configurable than that, this meets the majority of my needs.  Aside from the policy below, I was thinking I'd also have an hourly, weekly, and monthly backup policy.

I do see the copy-tags, but do not see the ability to add a new tag (though I see in tag.py the utility is intended to be used pretty globally).  Am I missing something by chance, or is this functionality that would need to be added?  

```
policies:
  - name: ec2-daily-backup
    resource: ec2
    filters:
       - type: value
        key: "tag:custodian-backup"
        op: "regex"
        value: ".*daily.*"
     actions:
       - type: snapshot
         copy-tags:
         - Name
         - CostCenter
         - POC
         - Service
         tag:                     <------ something like this?
           key: "Expiration"
           value: timestamp + 7 days logic (don't have that syntax just yet)
```

Thanks, Carrie


Discussion:
@caversa caversa
Hi Carrie, 
          The current method you are using would not work as its not supported.  What about if you copied that backup tag from the EC2 over to the snapshot (like the policy below) and then you could use the backup tag from a custodian ebs-snapshot policy like the one below.  Let me know if I am way off. :)

- Jamison
```
- name: ec2-daily-backup
    resource: ec2
    filters:
       - type: value
         key: "tag:custodian-backup"
         op: "regex"
         value: ".*daily.*"
     actions:
       - type: snapshot
         copy-tags:
         - Name
         - CostCenter
         - POC
         - Service
         - custodian-backup


- name: delete-old-daily-snapshots
  resource: ebs-snapshot
  filters:
        - type: age
          days: 7
          op: ge
        - type: value
          key: Description
          op: regex
          value: ^(Automated,Backup,-?)\w+
        - type: value
           key: "tag:custodian-backup"
           op: "regex"
           value: ".*daily.*"
  actions:
   - delete

```
Hi Jamison,

Thanks for that thoughtful answer; I do think that would work.  However, we use the Expiration tag across resources and it is a widely used convention within our company, so I was hoping to tie this in with the policies we already use and set the Expiration tag as the snapshot was created.

I'll mull this over - I may choose to add that 'add tag' feature on my own, or let the snapshots continue to be created via their cron jobs in order to add whatever tags we need.  Has anyone else expressed interest in having that as a feature, where it would benefit others to have this as well?  It may influence my decision on which option I choose.

Thanks, Carrie
thats interesting re expiration tag, thanks for sharing, we could probably support something like that as a feature addition. what sort of value do you typically do for an expiration tag.
Hi Kapil, I missed your question earlier.   We use a UTC formatted date, such as '2018-01-05 14:01:02 UTC'.   
For @jtroberts83 's suggestion (in case other people are considering this as well).  I think in many use cases, the same EC2 instance is backed up with different frequencies, such as both daily and weekly.  So if this is the case, you'd need two different custodian tags on the instance to indicate the backups.  Then you'd only copy the one tag down to the snapshot so that you aren't deleting the wrong snapshots.

```
policies:
  - name: ec2-daily-backup
    resource: ec2
    filters:
      - "tag:custodian-daily-backup": present
    actions:
       - type: snapshot
         copy-tags:
         - Name
         - CostCenter
         - custodian-daily-backup

  - name: ec2-weekly-backup
    resource: ec2
    filters:
      - "tag:custodian-weekly-backup": present
    actions:
       - type: snapshot
         copy-tags:
         - Name
         - CostCenter
         - custodian-weekly-backup
```

I was envisoning a single tag  that would indicate the frequencies of the backup, but this is doable for a workaround.


Was looking at using c7n for something very similar. I'm in the same situation where we are writing cron'd scripts to have the ec2 instance itself make its own snapshots. We currently do not do any smart automated cleanup of snapshots.

I was thinking of tagging the EBS volumes themselves, since I have an easy entry point for that in Terraform. Use tags on the volume to say it opts-in to snapshots and the desired snapshot frequency and retention. c7n snapshots and tags the snapshot with an opt-in and expiration timestamp. Another c7n policy to pull in expired snapshots and delete.

I'm more concerned at starting the process at the EBS volumes instead of the EC2 instance, but a lot of similarity. We have existing wild-west objects in AWS, which is why I want to be very paranoid and have opt-in tags on everything.

Resolving patch:
diff --git a/c7n/resources/ebs.py b/c7n/resources/ebs.py
--- a/c7n/resources/ebs.py
+++ b/c7n/resources/ebs.py
@@ -1150,26 +1150,55 @@ class CreateSnapshot(BaseAction):
                   - Attachments: []
                   - State: available
                 actions:
-                  - snapshot
+                  - type: snapshot
+                    copy-tags:
+                      - Name
     """
-    permissions = ('ec2:CreateSnapshot',)
-    schema = type_schema('snapshot')
+    schema = type_schema(
+        'snapshot',
+        **{'copy-tags': {'type': 'array', 'items': {'type': 'string'}},
+           'copy-volume-tags': {'type': 'boolean'}})
+    permissions = ('ec2:CreateSnapshot', 'ec2:CreateTags',)
+
+    def validate(self):
+        if self.data.get('copy-tags') and 'copy-volume-tags' in self.data:
+            raise PolicyValidationError(
+                "Can specify copy-tags or copy-volume-tags, not both")
 
     def process(self, volumes):
         client = local_session(self.manager.session_factory).client('ec2')
         retry = get_retry(['Throttled'], max_attempts=5)
         for vol in volumes:
             vol_id = vol['VolumeId']
-            retry(self.process_volume, client=client, volume=vol_id)
+            tags = [{
+                'ResourceType': 'snapshot',
+                'Tags': self.get_snapshot_tags(vol)
+            }]
+            retry(self.process_volume, client=client, volume=vol_id, tags=tags)
 
-    def process_volume(self, client, volume):
+    def process_volume(self, client, volume, tags):
         try:
-            client.create_snapshot(VolumeId=volume)
+            client.create_snapshot(VolumeId=volume, TagSpecifications=tags)
         except ClientError as e:
             if e.response['Error']['Code'] == 'InvalidVolume.NotFound':
                 return
             raise
 
+    def get_snapshot_tags(self, resource):
+        tags = [
+            {'Key': 'custodian_snapshot', 'Value': ''}]
+        copy_keys = self.data.get('copy-tags', [])
+        copy_tags = []
+        if copy_keys:
+            for t in resource.get('Tags', []):
+                if t['Key'] in copy_keys:
+                    copy_tags.append(t)
+            tags.extend(copy_tags)
+            return tags
+        if self.data.get('copy-volume-tags', True):
+            tags.extend(resource.get('Tags', []))
+        return tags
+
 
 @EBS.action_registry.register('delete')
 class Delete(BaseAction):


Test patch:
diff --git a/tests/data/placebo/test_ebs_snapshot_copy_tags/ec2.CreateSnapshot_1.json b/tests/data/placebo/test_ebs_snapshot_copy_tags/ec2.CreateSnapshot_1.json
new file mode 100644
--- /dev/null
+++ b/tests/data/placebo/test_ebs_snapshot_copy_tags/ec2.CreateSnapshot_1.json
@@ -0,0 +1,49 @@
+{
+  "status_code": 200,
+  "data": {
+    "Description": "",
+    "ResponseMetadata": {
+      "RetryAttempts": 0,
+      "HTTPStatusCode": 200,
+      "RequestId": "526a3e1a-2002-4ec6-8860-cc09008173ac",
+      "HTTPHeaders": {
+        "transfer-encoding": "chunked",
+        "vary": "Accept-Encoding",
+        "server": "AmazonEC2",
+        "content-type": "text/xml;charset=UTF-8",
+        "date": "Wed, 03 May 2017 14:43:16 GMT"
+      }
+    },
+    "Encrypted": false,
+    "VolumeId": "vol-0252f61378ede9d01",
+    "State": "pending",
+    "VolumeSize": 8,
+    "Progress": "",
+    "StartTime": {
+      "hour": 14,
+      "__class__": "datetime",
+      "month": 5,
+      "second": 17,
+      "microsecond": 0,
+      "year": 2017,
+      "day": 3,
+      "minute": 43
+    },
+    "SnapshotId": "snap-0c8b611065fed033e",
+    "Tags": [
+      {
+        "Value": "FancyTestInstancePrivate",
+        "Key": "Name"
+      },
+      {
+        "Key": "Stage",
+        "Value": "Dev"
+      },
+      {
+        "Key": "custodian_snapshot",
+        "Value": ""
+      }
+    ],
+    "OwnerId": "644160558196"
+  }
+}
diff --git a/tests/data/placebo/test_ebs_snapshot_copy_tags/ec2.DescribeSnapshots_1.json b/tests/data/placebo/test_ebs_snapshot_copy_tags/ec2.DescribeSnapshots_1.json
new file mode 100644
--- /dev/null
+++ b/tests/data/placebo/test_ebs_snapshot_copy_tags/ec2.DescribeSnapshots_1.json
@@ -0,0 +1,53 @@
+{
+  "status_code": 200,
+  "data": {
+    "ResponseMetadata": {
+      "RetryAttempts": 0,
+      "HTTPStatusCode": 200,
+      "RequestId": "48383548-8c6e-47de-815e-33f0ebe7f580",
+      "HTTPHeaders": {
+        "transfer-encoding": "chunked",
+        "vary": "Accept-Encoding",
+        "server": "AmazonEC2",
+        "content-type": "text/xml;charset=UTF-8",
+        "date": "Wed, 03 May 2017 14:43:17 GMT"
+      }
+    },
+    "Snapshots": [
+      {
+        "Description": "",
+        "Encrypted": false,
+        "VolumeId": "vol-0252f61378ede9d01",
+        "State": "pending",
+        "VolumeSize": 8,
+        "Progress": "99%",
+        "StartTime": {
+          "hour": 14,
+          "__class__": "datetime",
+          "month": 5,
+          "second": 17,
+          "microsecond": 0,
+          "year": 2017,
+          "day": 3,
+          "minute": 43
+        },
+        "SnapshotId": "snap-0c8b611065fed033e",
+        "Tags": [
+          {
+            "Value": "FancyTestInstancePrivate",
+            "Key": "Name"
+          },
+          {
+            "Key": "Stage",
+            "Value": "Dev"
+          },
+          {
+            "Key": "custodian_snapshot",
+            "Value": ""
+          }
+        ],
+        "OwnerId": "644160558196"
+      }
+    ]
+  }
+}
\ No newline at end of file
diff --git a/tests/data/placebo/test_ebs_snapshot_copy_tags/ec2.DescribeVolumes_1.json b/tests/data/placebo/test_ebs_snapshot_copy_tags/ec2.DescribeVolumes_1.json
new file mode 100644
--- /dev/null
+++ b/tests/data/placebo/test_ebs_snapshot_copy_tags/ec2.DescribeVolumes_1.json
@@ -0,0 +1,156 @@
+{
+  "status_code": 200,
+  "data": {
+    "ResponseMetadata": {
+      "RetryAttempts": 0,
+      "HTTPStatusCode": 200,
+      "RequestId": "53e62ac7-14d4-4657-9385-e1ffeb34edea",
+      "HTTPHeaders": {
+        "transfer-encoding": "chunked",
+        "vary": "Accept-Encoding",
+        "server": "AmazonEC2",
+        "content-type": "text/xml;charset=UTF-8",
+        "date": "Wed, 03 May 2017 14:43:16 GMT"
+      }
+    },
+    "Volumes": [
+      {
+        "AvailabilityZone": "us-east-1b",
+        "Attachments": [
+          {
+            "AttachTime": {
+              "hour": 14,
+              "__class__": "datetime",
+              "month": 3,
+              "second": 28,
+              "microsecond": 0,
+              "year": 2017,
+              "day": 28,
+              "minute": 55
+            },
+            "InstanceId": "i-0a0b51bcf11a8cdfb",
+            "VolumeId": "vol-01adbb6a4f175941d",
+            "State": "attached",
+            "DeleteOnTermination": true,
+            "Device": "/dev/xvda"
+          }
+        ],
+        "Encrypted": false,
+        "VolumeType": "gp2",
+        "VolumeId": "vol-01adbb6a4f175941d",
+        "State": "in-use",
+        "Iops": 100,
+        "SnapshotId": "snap-037f1f9e6c8ea4d65",
+        "CreateTime": {
+          "hour": 14,
+          "__class__": "datetime",
+          "month": 3,
+          "second": 28,
+          "microsecond": 486000,
+          "year": 2017,
+          "day": 28,
+          "minute": 55
+        },
+        "Size": 8
+      },
+      {
+        "AvailabilityZone": "us-east-1b",
+        "Attachments": [
+          {
+            "AttachTime": {
+              "hour": 20,
+              "__class__": "datetime",
+              "month": 4,
+              "second": 21,
+              "microsecond": 0,
+              "year": 2017,
+              "day": 10,
+              "minute": 41
+            },
+            "InstanceId": "i-0be2fdca058278ea3",
+            "VolumeId": "vol-0252f61378ede9d01",
+            "State": "attached",
+            "DeleteOnTermination": true,
+            "Device": "/dev/sda1"
+          }
+        ],
+        "Tags": [
+          {
+            "Value": "FancyTestInstancePrivate",
+            "Key": "Name"
+          }
+        ],
+        "Encrypted": false,
+        "VolumeType": "gp2",
+        "VolumeId": "vol-0252f61378ede9d01",
+        "State": "in-use",
+        "Iops": 100,
+        "SnapshotId": "snap-00762b50998e9113b",
+        "CreateTime": {
+          "hour": 20,
+          "__class__": "datetime",
+          "month": 4,
+          "second": 21,
+          "microsecond": 977000,
+          "year": 2017,
+          "day": 10,
+          "minute": 41
+        },
+        "Size": 8,
+        "Tags": [
+          {
+            "Value": "FancyTestInstancePrivate",
+            "Key": "Name"
+          },
+          {
+            "Key": "Stage",
+            "Value": "Dev"
+          },
+          {
+            "Key": "DoNotCopy",
+            "Value": "ThisTag"
+          }
+        ]
+      },
+      {
+        "AvailabilityZone": "us-east-1d",
+        "Attachments": [
+          {
+            "AttachTime": {
+              "hour": 1,
+              "__class__": "datetime",
+              "month": 4,
+              "second": 40,
+              "microsecond": 0,
+              "year": 2017,
+              "day": 3,
+              "minute": 34
+            },
+            "InstanceId": "i-0aace5764d21d3105",
+            "VolumeId": "vol-0b5bc31549f55bdd2",
+            "State": "attached",
+            "DeleteOnTermination": true,
+            "Device": "/dev/xvda"
+          }
+        ],
+        "Encrypted": false,
+        "VolumeType": "gp2",
+        "VolumeId": "vol-0b5bc31549f55bdd2",
+        "State": "in-use",
+        "Iops": 100,
+        "SnapshotId": "snap-037f1f9e6c8ea4d65",
+        "CreateTime": {
+          "hour": 1,
+          "__class__": "datetime",
+          "month": 4,
+          "second": 40,
+          "microsecond": 677000,
+          "year": 2017,
+          "day": 3,
+          "minute": 34
+        },
+        "Size": 8
+      }
+    ]
+  }
+}
\ No newline at end of file
diff --git a/tests/test_ebs.py b/tests/test_ebs.py
--- a/tests/test_ebs.py
+++ b/tests/test_ebs.py
@@ -421,6 +421,28 @@ def test_volume_snapshot(self):
         )
         self.assertEqual(len(snapshot_data["Snapshots"]), 1)
 
+    def test_volume_snapshot_copy_tags(self):
+        factory = self.replay_flight_data("test_ebs_snapshot_copy_tags")
+        policy = self.load_policy(
+            {
+                "name": "ebs-test-snapshot",
+                "resource": "ebs",
+                "filters": [{"VolumeId": "vol-0252f61378ede9d01"}],
+                "actions": [{"type": "snapshot", "copy-tags": ['Name', 'Stage']}]
+            },
+            session_factory=factory,
+        )
+        resources = policy.run()
+        self.assertEqual(len(resources), 1)
+        snapshot_data = factory().client("ec2").describe_snapshots(
+            Filters=[{"Name": "volume-id", "Values": ["vol-0252f61378ede9d01"]}]
+        )
+        rtags = {t['Key']: t['Value'] for t in resources[0]['Tags']}
+        rtags.pop('DoNotCopy')
+        rtags['custodian_snapshot'] = ''
+        for s in snapshot_data['Snapshots']:
+            self.assertEqual(rtags, {t['Key']: t['Value'] for t in s['Tags']})
+
 
 class VolumeDeleteTest(BaseTest):