Repository: cloud-custodian/cloud-custodian

Issue:
rds-cluster - Add Deletion Protection support
It appears that Deletion Protection support has not been added for rds-cluster resources.

Ideally, we could take similar actions that we do for the RDS resource. This would require that `modify-db-cluster` and `DeletionProtection` are added.

For example:
```
    actions:
      - type: modify-db-cluster
        update:
          - property: DeletionProtection
            value: false
        immediate: true
      - type: delete
        skip-snapshot: true
        delete-instances: true
```

When processing the DeleteDBCluster in the `rds-cluster > delete` workflow, I receive this error and an empty (and available) cluster remains: `An error occurred (InvalidParameterCombination) when calling the DeleteDBCluster operation: Cannot delete protected Cluster, please disable deletion protection and try again.`


Discussion:
sounds good
There can be a case where:

db-x: deletion protection=false
db-cluster-x: deletion protection=true
db-x is a part of db-cluster-x. 
In the above case, if we want to delete the cluster, we should have the ability to filter them. @kapilt , do we want to register a new resource called db-cluster? Thanks

@PratMis not sure what your asking, we already have an rds cluster resource (which is what this issue is about), and you can already filter them on delete protection.
Ah, Sorry! my bad! I didn't realize that there was a rdscluster.py file. I wrote the code and test case for this feature. I'm successfully able to filter out a cluster with a normal policy like the one shown below:
```
policies:
  - name: db-cluster-filter
    resource: rds-cluster
    filters:
      - DBClusterIdentifier: c7n-test-cluster
```
But running a test case with a similar filter give me a traceback:
```
    def test_modify_rds_cluster(self):
        session_factory = self.record_flight_data("test_modify_rds_cluster")
        p = self.load_policy(
            {
                "name": "modify-db-cluster",
                "resource": "rds-cluster",
                "filters": [{"DBClusterIdentifier": "c7n-test-cluster"}],
                "actions": [
                    {
                        "type": "modify-db-cluster",
                        "update": [
                            {
                                "property": 'DeletionProtection',
                                "value": False
                            }
                        ],
                        "immediate": True
                    }
                ],
            },
            session_factory=session_factory,
        )
        resources = p.run()
        self.assertEqual(len(resources), 1)
``` 
Traceback:
```
Traceback (most recent call last):
  File "/Users/Desktop/cloud-custodian/c7n/policy.py", line 232, in run
    resources = self.policy.resource_manager.resources()
  File "/Users/Desktop/cloud-custodian/c7n/query.py", line 426, in resources
    resources = self.augment(resources)
  File "/Users/Desktop/cloud-custodian/c7n/resources/rdscluster.py", line 65, in augment
    self.generate_arn, self.retry)))
  File "/Users/Desktop/cloud-custodian/c7n/resources/rdscluster.py", line 86, in _rds_cluster_tags
    return list(filter(None, map(process_tags, dbs)))
  File "/Users/Desktop/cloud-custodian/c7n/resources/rdscluster.py", line 80, in process_tags
    ResourceName=generator(db[model.id]))['TagList']
  File "/Users/Desktop/cloud-custodian/c7n/utils.py", line 375, in _retry
    return func(*args, **kw)
  File "/Users/Desktop/cloud-custodian/lib/python3.7/site-packages/botocore/client.py", line 357, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/Users/Desktop/cloud-custodian/lib/python3.7/site-packages/botocore/client.py", line 661, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (InvalidParameterValue) when calling the ListTagsForResource operation: Invalid resource name:  arn:aws:rds:us-east-1::cluster:c7n-test-cluster
```
On further investigation, looks like the resource arn is incorrect. Can't see the AWS account number in the arn. Thanks!
Might be my mistake. I'm still investigating. Will report back 

Resolving patch:
diff --git a/c7n/resources/rdscluster.py b/c7n/resources/rdscluster.py
--- a/c7n/resources/rdscluster.py
+++ b/c7n/resources/rdscluster.py
@@ -1,4 +1,4 @@
-# Copyright 2016-2017 Capital One Services, LLC
+# Copyright 2016-2019 Capital One Services, LLC
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -24,6 +24,8 @@
 from c7n.manager import resources
 from c7n.query import QueryResourceManager
 from c7n import tags
+from .aws import shape_validate
+from c7n.exceptions import PolicyValidationError
 from c7n.utils import (
     type_schema, local_session, snapshot_identifier, chunks,
     get_retry, generate_arn)
@@ -399,6 +401,55 @@ def process(self, clusters):
                 client.exceptions.InvalidDBClusterStateFault)
 
 
+@RDSCluster.action_registry.register('modify-db-cluster')
+class ModifyDbCluster(BaseAction):
+    """Modifies an RDS instance based on specified parameter
+    using ModifyDbInstance.
+
+    'Immediate" determines whether the modification is applied immediately
+    or not. If 'immediate' is not specified, default is false.
+
+    :example:
+
+    .. code-block:: yaml
+
+            policies:
+              - name: disable-db-cluster-deletion-protection
+                resource: rds-cluster
+                filters:
+                  - DeletionProtection: true
+                  - PubliclyAccessible: true
+                actions:
+                  - type: modify-db-cluster
+                    attributes:
+                        CopyTagsToSnapshot: true
+                        DeletionProtection: false
+    """
+
+    schema = type_schema(
+        'modify-db-cluster',
+        attributes={'type': 'object'},
+        required=('attributes',))
+
+    permissions = ('rds:ModifyDBCluster',)
+    shape = 'ModifyDBClusterMessage'
+
+    def validate(self):
+        attrs = dict(self.data['attributes'])
+        if 'DBClusterIdentifier' in attrs:
+            raise PolicyValidationError(
+                "Can't include DBClusterIdentifier in modify-db-cluster action")
+        attrs['DBClusterIdentifier'] = 'PolicyValidation'
+        return shape_validate(attrs, self.shape, 'rds')
+
+    def process(self, clusters):
+        client = local_session(self.manager.session_factory).client('rds')
+        for c in clusters:
+            client.modify_db_cluster(
+                DBClusterIdentifier=c['DBClusterIdentifier'],
+                **self.data['attributes'])
+
+
 @resources.register('rds-cluster-snapshot')
 class RDSClusterSnapshot(QueryResourceManager):
     """Resource manager for RDS cluster snapshots.


Test patch:
diff --git a/tests/data/placebo/test_modify_rds_cluster/rds.DescribeDBClusters_1.json b/tests/data/placebo/test_modify_rds_cluster/rds.DescribeDBClusters_1.json
new file mode 100644
--- /dev/null
+++ b/tests/data/placebo/test_modify_rds_cluster/rds.DescribeDBClusters_1.json
@@ -0,0 +1,127 @@
+{
+    "status_code": 200,
+    "data": {
+        "DBClusters": [
+            {
+                "AllocatedStorage": 1,
+                "AvailabilityZones": [
+                    "us-east-1d",
+                    "us-east-1a",
+                    "us-east-1b"
+                ],
+                "BackupRetentionPeriod": 1,
+                "DatabaseName": "mytest",
+                "DBClusterIdentifier": "mytest",
+                "DBClusterParameterGroup": "default.aurora5.6",
+                "DBSubnetGroup": "default",
+                "Status": "available",
+                "EarliestRestorableTime": {
+                    "__class__": "datetime",
+                    "year": 2019,
+                    "month": 3,
+                    "day": 20,
+                    "hour": 9,
+                    "minute": 31,
+                    "second": 32,
+                    "microsecond": 869000
+                },
+                "Endpoint": "mytest.cluster-cz6wy8cci3uf.us-east-1.rds.amazonaws.com",
+                "ReaderEndpoint": "mytest.cluster-ro-cz6wy8cci3uf.us-east-1.rds.amazonaws.com",
+                "MultiAZ": true,
+                "Engine": "aurora",
+                "EngineVersion": "5.6.10a",
+                "LatestRestorableTime": {
+                    "__class__": "datetime",
+                    "year": 2019,
+                    "month": 3,
+                    "day": 22,
+                    "hour": 6,
+                    "minute": 20,
+                    "second": 19,
+                    "microsecond": 796000
+                },
+                "Port": 3306,
+                "MasterUsername": "mytest",
+                "PreferredBackupWindow": "09:27-09:57",
+                "PreferredMaintenanceWindow": "tue:06:41-tue:07:11",
+                "ReadReplicaIdentifiers": [],
+                "DBClusterMembers": [
+                    {
+                        "DBInstanceIdentifier": "mycluster-us-east-1a",
+                        "IsClusterWriter": false,
+                        "DBClusterParameterGroupStatus": "in-sync",
+                        "PromotionTier": 1
+                    },
+                    {
+                        "DBInstanceIdentifier": "mycluster",
+                        "IsClusterWriter": true,
+                        "DBClusterParameterGroupStatus": "in-sync",
+                        "PromotionTier": 1
+                    }
+                ],
+                "VpcSecurityGroups": [
+                    {
+                        "VpcSecurityGroupId": "sg-02ec691909b2a4753",
+                        "Status": "active"
+                    },
+                    {
+                        "VpcSecurityGroupId": "sg-6c7fa917",
+                        "Status": "active"
+                    },
+                    {
+                        "VpcSecurityGroupId": "sg-08ee699c0c4e168e4",
+                        "Status": "active"
+                    }
+                ],
+                "HostedZoneId": "Z2R2ITUGPM61AM",
+                "StorageEncrypted": true,
+                "KmsKeyId": "arn:aws:kms:us-east-1:644160558196:key/b10f842a-feb7-4318-92d5-0640a75b7688",
+                "DbClusterResourceId": "cluster-FLECGNGAS7PCRHRT7DHDF24BGU",
+                "DBClusterArn": "arn:aws:rds:us-east-1:644160558196:cluster:mytest",
+                "AssociatedRoles": [],
+                "IAMDatabaseAuthenticationEnabled": true,
+                "ClusterCreateTime": {
+                    "__class__": "datetime",
+                    "year": 2019,
+                    "month": 2,
+                    "day": 19,
+                    "hour": 15,
+                    "minute": 33,
+                    "second": 56,
+                    "microsecond": 465000
+                },
+                "EarliestBacktrackTime": {
+                    "__class__": "datetime",
+                    "year": 2019,
+                    "month": 3,
+                    "day": 21,
+                    "hour": 22,
+                    "minute": 22,
+                    "second": 46,
+                    "microsecond": 807000
+                },
+                "BacktrackWindow": 28800,
+                "BacktrackConsumedChangeRecords": 1647625,
+                "EnabledCloudwatchLogsExports": [
+                    "slowquery"
+                ],
+                "EngineMode": "provisioned",
+                "DeletionProtection": true,
+                "HttpEndpointEnabled": false,
+                "CopyTagsToSnapshot": false
+            }
+        ],
+        "ResponseMetadata": {
+            "RequestId": "debe38e2-1fb1-4d9d-b918-79ce9cdc15c7",
+            "HTTPStatusCode": 200,
+            "HTTPHeaders": {
+                "x-amzn-requestid": "debe38e2-1fb1-4d9d-b918-79ce9cdc15c7",
+                "content-type": "text/xml",
+                "content-length": "4111",
+                "vary": "Accept-Encoding",
+                "date": "Fri, 22 Mar 2019 06:23:19 GMT"
+            },
+            "RetryAttempts": 0
+        }
+    }
+}
\ No newline at end of file
diff --git a/tests/data/placebo/test_modify_rds_cluster/rds.DescribeDBClusters_2.json b/tests/data/placebo/test_modify_rds_cluster/rds.DescribeDBClusters_2.json
new file mode 100644
--- /dev/null
+++ b/tests/data/placebo/test_modify_rds_cluster/rds.DescribeDBClusters_2.json
@@ -0,0 +1,127 @@
+{
+    "status_code": 200,
+    "data": {
+        "DBClusters": [
+            {
+                "AllocatedStorage": 1,
+                "AvailabilityZones": [
+                    "us-east-1d",
+                    "us-east-1a",
+                    "us-east-1b"
+                ],
+                "BackupRetentionPeriod": 1,
+                "DatabaseName": "mytest",
+                "DBClusterIdentifier": "mytest",
+                "DBClusterParameterGroup": "default.aurora5.6",
+                "DBSubnetGroup": "default",
+                "Status": "available",
+                "EarliestRestorableTime": {
+                    "__class__": "datetime",
+                    "year": 2019,
+                    "month": 3,
+                    "day": 20,
+                    "hour": 9,
+                    "minute": 31,
+                    "second": 32,
+                    "microsecond": 869000
+                },
+                "Endpoint": "mytest.cluster-cz6wy8cci3uf.us-east-1.rds.amazonaws.com",
+                "ReaderEndpoint": "mytest.cluster-ro-cz6wy8cci3uf.us-east-1.rds.amazonaws.com",
+                "MultiAZ": true,
+                "Engine": "aurora",
+                "EngineVersion": "5.6.10a",
+                "LatestRestorableTime": {
+                    "__class__": "datetime",
+                    "year": 2019,
+                    "month": 3,
+                    "day": 22,
+                    "hour": 6,
+                    "minute": 20,
+                    "second": 19,
+                    "microsecond": 796000
+                },
+                "Port": 3306,
+                "MasterUsername": "mytest",
+                "PreferredBackupWindow": "09:27-09:57",
+                "PreferredMaintenanceWindow": "tue:06:41-tue:07:11",
+                "ReadReplicaIdentifiers": [],
+                "DBClusterMembers": [
+                    {
+                        "DBInstanceIdentifier": "mycluster-us-east-1a",
+                        "IsClusterWriter": false,
+                        "DBClusterParameterGroupStatus": "in-sync",
+                        "PromotionTier": 1
+                    },
+                    {
+                        "DBInstanceIdentifier": "mycluster",
+                        "IsClusterWriter": true,
+                        "DBClusterParameterGroupStatus": "in-sync",
+                        "PromotionTier": 1
+                    }
+                ],
+                "VpcSecurityGroups": [
+                    {
+                        "VpcSecurityGroupId": "sg-02ec691909b2a4753",
+                        "Status": "active"
+                    },
+                    {
+                        "VpcSecurityGroupId": "sg-6c7fa917",
+                        "Status": "active"
+                    },
+                    {
+                        "VpcSecurityGroupId": "sg-08ee699c0c4e168e4",
+                        "Status": "active"
+                    }
+                ],
+                "HostedZoneId": "Z2R2ITUGPM61AM",
+                "StorageEncrypted": true,
+                "KmsKeyId": "arn:aws:kms:us-east-1:644160558196:key/b10f842a-feb7-4318-92d5-0640a75b7688",
+                "DbClusterResourceId": "cluster-FLECGNGAS7PCRHRT7DHDF24BGU",
+                "DBClusterArn": "arn:aws:rds:us-east-1:644160558196:cluster:mytest",
+                "AssociatedRoles": [],
+                "IAMDatabaseAuthenticationEnabled": true,
+                "ClusterCreateTime": {
+                    "__class__": "datetime",
+                    "year": 2019,
+                    "month": 2,
+                    "day": 19,
+                    "hour": 15,
+                    "minute": 33,
+                    "second": 56,
+                    "microsecond": 465000
+                },
+                "EarliestBacktrackTime": {
+                    "__class__": "datetime",
+                    "year": 2019,
+                    "month": 3,
+                    "day": 21,
+                    "hour": 22,
+                    "minute": 22,
+                    "second": 46,
+                    "microsecond": 807000
+                },
+                "BacktrackWindow": 28800,
+                "BacktrackConsumedChangeRecords": 1647625,
+                "EnabledCloudwatchLogsExports": [
+                    "slowquery"
+                ],
+                "EngineMode": "provisioned",
+                "DeletionProtection": false,
+                "HttpEndpointEnabled": false,
+                "CopyTagsToSnapshot": false
+            }
+        ],
+        "ResponseMetadata": {
+            "RequestId": "a753e0cb-b1ff-4483-ac3d-0c73c3c0c8bc",
+            "HTTPStatusCode": 200,
+            "HTTPHeaders": {
+                "x-amzn-requestid": "a753e0cb-b1ff-4483-ac3d-0c73c3c0c8bc",
+                "content-type": "text/xml",
+                "content-length": "4112",
+                "vary": "Accept-Encoding",
+                "date": "Fri, 22 Mar 2019 06:23:21 GMT"
+            },
+            "RetryAttempts": 0
+        }
+    }
+}
\ No newline at end of file
diff --git a/tests/data/placebo/test_modify_rds_cluster/rds.ListTagsForResource_1.json b/tests/data/placebo/test_modify_rds_cluster/rds.ListTagsForResource_1.json
new file mode 100644
--- /dev/null
+++ b/tests/data/placebo/test_modify_rds_cluster/rds.ListTagsForResource_1.json
@@ -0,0 +1,22 @@
+{
+    "status_code": 200,
+    "data": {
+        "TagList": [
+            {
+                "Key": "Owner",
+                "Value": "kapil"
+            }
+        ],
+        "ResponseMetadata": {
+            "RequestId": "0d409697-0534-4758-8cdc-c48aed51e17b",
+            "HTTPStatusCode": 200,
+            "HTTPHeaders": {
+                "x-amzn-requestid": "0d409697-0534-4758-8cdc-c48aed51e17b",
+                "content-type": "text/xml",
+                "content-length": "386",
+                "date": "Fri, 22 Mar 2019 06:23:20 GMT"
+            },
+            "RetryAttempts": 0
+        }
+    }
+}
\ No newline at end of file
diff --git a/tests/data/placebo/test_modify_rds_cluster/rds.ModifyDBCluster_1.json b/tests/data/placebo/test_modify_rds_cluster/rds.ModifyDBCluster_1.json
new file mode 100644
--- /dev/null
+++ b/tests/data/placebo/test_modify_rds_cluster/rds.ModifyDBCluster_1.json
@@ -0,0 +1,125 @@
+{
+    "status_code": 200,
+    "data": {
+        "DBCluster": {
+            "AllocatedStorage": 1,
+            "AvailabilityZones": [
+                "us-east-1d",
+                "us-east-1a",
+                "us-east-1b"
+            ],
+            "BackupRetentionPeriod": 1,
+            "DatabaseName": "mytest",
+            "DBClusterIdentifier": "mytest",
+            "DBClusterParameterGroup": "default.aurora5.6",
+            "DBSubnetGroup": "default",
+            "Status": "available",
+            "EarliestRestorableTime": {
+                "__class__": "datetime",
+                "year": 2019,
+                "month": 3,
+                "day": 20,
+                "hour": 9,
+                "minute": 31,
+                "second": 32,
+                "microsecond": 869000
+            },
+            "Endpoint": "mytest.cluster-cz6wy8cci3uf.us-east-1.rds.amazonaws.com",
+            "ReaderEndpoint": "mytest.cluster-ro-cz6wy8cci3uf.us-east-1.rds.amazonaws.com",
+            "MultiAZ": true,
+            "Engine": "aurora",
+            "EngineVersion": "5.6.10a",
+            "LatestRestorableTime": {
+                "__class__": "datetime",
+                "year": 2019,
+                "month": 3,
+                "day": 22,
+                "hour": 6,
+                "minute": 20,
+                "second": 19,
+                "microsecond": 796000
+            },
+            "Port": 3306,
+            "MasterUsername": "mytest",
+            "PreferredBackupWindow": "09:27-09:57",
+            "PreferredMaintenanceWindow": "tue:06:41-tue:07:11",
+            "ReadReplicaIdentifiers": [],
+            "DBClusterMembers": [
+                {
+                    "DBInstanceIdentifier": "mycluster-us-east-1a",
+                    "IsClusterWriter": false,
+                    "DBClusterParameterGroupStatus": "in-sync",
+                    "PromotionTier": 1
+                },
+                {
+                    "DBInstanceIdentifier": "mycluster",
+                    "IsClusterWriter": true,
+                    "DBClusterParameterGroupStatus": "in-sync",
+                    "PromotionTier": 1
+                }
+            ],
+            "VpcSecurityGroups": [
+                {
+                    "VpcSecurityGroupId": "sg-02ec691909b2a4753",
+                    "Status": "active"
+                },
+                {
+                    "VpcSecurityGroupId": "sg-6c7fa917",
+                    "Status": "active"
+                },
+                {
+                    "VpcSecurityGroupId": "sg-08ee699c0c4e168e4",
+                    "Status": "active"
+                }
+            ],
+            "HostedZoneId": "Z2R2ITUGPM61AM",
+            "StorageEncrypted": true,
+            "KmsKeyId": "arn:aws:kms:us-east-1:644160558196:key/b10f842a-feb7-4318-92d5-0640a75b7688",
+            "DbClusterResourceId": "cluster-FLECGNGAS7PCRHRT7DHDF24BGU",
+            "DBClusterArn": "arn:aws:rds:us-east-1:644160558196:cluster:mytest",
+            "AssociatedRoles": [],
+            "IAMDatabaseAuthenticationEnabled": true,
+            "ClusterCreateTime": {
+                "__class__": "datetime",
+                "year": 2019,
+                "month": 2,
+                "day": 19,
+                "hour": 15,
+                "minute": 33,
+                "second": 56,
+                "microsecond": 465000
+            },
+            "EarliestBacktrackTime": {
+                "__class__": "datetime",
+                "year": 2019,
+                "month": 3,
+                "day": 21,
+                "hour": 22,
+                "minute": 22,
+                "second": 46,
+                "microsecond": 807000
+            },
+            "BacktrackWindow": 28800,
+            "BacktrackConsumedChangeRecords": 1647625,
+            "EnabledCloudwatchLogsExports": [
+                "slowquery"
+            ],
+            "EngineMode": "provisioned",
+            "DeletionProtection": false,
+            "HttpEndpointEnabled": false,
+            "CopyTagsToSnapshot": false
+        },
+        "ResponseMetadata": {
+            "RequestId": "5a48770c-6776-4c61-b1aa-150d6cf5c040",
+            "HTTPStatusCode": 200,
+            "HTTPHeaders": {
+                "x-amzn-requestid": "5a48770c-6776-4c61-b1aa-150d6cf5c040",
+                "content-type": "text/xml",
+                "content-length": "3856",
+                "vary": "Accept-Encoding",
+                "date": "Fri, 22 Mar 2019 06:23:20 GMT"
+            },
+            "RetryAttempts": 0
+        }
+    }
+}
\ No newline at end of file
diff --git a/tests/test_rdscluster.py b/tests/test_rdscluster.py
--- a/tests/test_rdscluster.py
+++ b/tests/test_rdscluster.py
@@ -158,6 +158,29 @@ def test_rdscluster_snapshot(self):
         resources = p.run()
         self.assertEqual(len(resources), 1)
 
+    def test_modify_rds_cluster(self):
+        session_factory = self.replay_flight_data("test_modify_rds_cluster")
+        p = self.load_policy(
+            {
+                "name": "modify-db-cluster",
+                "resource": "rds-cluster",
+                "filters": [{"DeletionProtection": True}],
+                "actions": [{
+                    "type": "modify-db-cluster",
+                    "attributes": {
+                        "DeletionProtection": False}
+                }]
+            },
+            session_factory=session_factory, config={'account_id': '644160558196'}
+        )
+        resources = p.run()
+        self.assertEqual(len(resources), 1)
+
+        client = session_factory().client("rds")
+        cluster = client.describe_db_clusters(
+            DBClusterIdentifier='mytest')
+        self.assertFalse(cluster['DBClusters'][0]['DeletionProtection'])
+
     def test_rdscluster_tag_augment(self):
         session_factory = self.replay_flight_data("test_rdscluster_tag_augment")
         p = self.load_policy(