Repository: cloud-custodian/cloud-custodian

Issue:
Key error when adding a policy statement to an SNS Topic
### Describe the bug

When adding/describing a statement to an SNS topic a key error is thrown when attempting to retrieve the `Sid` of the policy.

### What did you expect to happen?

A policy statement should be described and added without error

### Cloud Provider

Amazon Web Services (AWS)

### Cloud Custodian version and dependency information

```shell
Custodian:   0.9.13
Python:      3.8.11 (default, Jun 29 2021, 03:17:05) 
             [Clang 12.0.0 (clang-1200.0.32.29)]
Platform:    posix.uname_result(sysname='Darwin', nodename='f8ffc250c2ee', release='20.6.0', version='Darwin Kernel Version 20.6.0: Mon Aug 30 06:12:21 PDT 2021; root:xnu-7195.141.6~3/RELEASE_X86_64', machine='x86_64')
Using venv:  True
Docker: False
Installed: 

PyJWT==1.7.1
PyYAML==5.4.1
adal==1.2.7
appdirs==1.4.4
applicationinsights==0.11.10
apscheduler==3.7.0
argcomplete==1.12.3
attrs==21.2.0
azure-common==1.1.27
azure-core==1.15.0
azure-cosmos==3.2.0
azure-cosmosdb-nspkg==2.0.2
azure-cosmosdb-table==1.0.6
azure-functions==1.7.2
azure-graphrbac==0.61.1
azure-identity==1.6.0
azure-keyvault==4.1.0
azure-keyvault-certificates==4.3.0
azure-keyvault-keys==4.4.0
azure-keyvault-secrets==4.3.0
azure-mgmt-apimanagement==1.0.0
azure-mgmt-applicationinsights==1.0.0
azure-mgmt-authorization==1.0.0
azure-mgmt-batch==15.0.0
azure-mgmt-cdn==10.0.0
azure-mgmt-cognitiveservices==11.0.0
azure-mgmt-compute==19.0.0
azure-mgmt-containerinstance==7.0.0
azure-mgmt-containerregistry==8.0.0b1
azure-mgmt-containerservice==15.1.0
azure-mgmt-core==1.2.2
azure-mgmt-cosmosdb==6.4.0
azure-mgmt-costmanagement==1.0.0
azure-mgmt-databricks==1.0.0b1
azure-mgmt-datafactory==1.1.0
azure-mgmt-datalake-store==1.0.0
azure-mgmt-dns==8.0.0b1
azure-mgmt-eventgrid==8.0.0
azure-mgmt-eventhub==8.0.0
azure-mgmt-frontdoor==1.0.0
azure-mgmt-hdinsight==7.0.0
azure-mgmt-iothub==1.0.0
azure-mgmt-keyvault==8.0.0
azure-mgmt-logic==9.0.0
azure-mgmt-managementgroups==1.0.0b1
azure-mgmt-monitor==2.0.0
azure-mgmt-msi==1.0.0
azure-mgmt-network==17.1.0
azure-mgmt-policyinsights==1.0.0
azure-mgmt-rdbms==8.1.0
azure-mgmt-redis==12.0.0
azure-mgmt-resource==16.1.0
azure-mgmt-resourcegraph==7.0.0
azure-mgmt-search==8.0.0
azure-mgmt-servicefabric==1.0.0
azure-mgmt-sql==1.0.0
azure-mgmt-storage==17.1.0
azure-mgmt-subscription==1.0.0
azure-mgmt-trafficmanager==0.51.0
azure-mgmt-web==2.0.0
azure-nspkg==3.0.2
azure-storage-blob==12.8.1
azure-storage-common==2.1.0
azure-storage-file==2.1.0
azure-storage-file-share==12.5.0
azure-storage-queue==12.1.6
boto3==1.17.102
botocore==1.20.102
c7n==0.9.13
cachetools==4.2.2
certifi==2021.5.30
cffi==1.14.5
chardet==4.0.0
click==7.1.2
cryptography==3.4.7
decorator==5.0.9
distlib==0.3.2
dogpile.cache==1.1.3
google-api-core==1.30.0
google-api-python-client==1.12.8
google-auth==1.32.0
google-auth-httplib2==0.1.0
google-cloud-core==1.7.1
google-cloud-logging==1.15.1
google-cloud-monitoring==0.34.0
google-cloud-storage==1.40.0
google-crc32c==1.1.2
google-resumable-media==1.3.1
googleapis-common-protos==1.53.0
httplib2==0.19.1
idna==2.10
importlib-metadata==4.6.0
iso8601==0.1.14
isodate==0.6.0
jmespath==0.10.0
jsonpatch==1.32
jsonpointer==2.1
jsonschema==3.2.0
keystoneauth1==4.3.1
kubernetes==10.0.1
mock==4.0.3
msal==1.12.0
msal-extensions==0.3.0
msrest==0.6.21
msrestazure==0.6.4
munch==2.5.0
netaddr==0.7.20
netifaces==0.11.0
oauthlib==3.1.1
openstacksdk==0.52.0
os-service-types==1.7.0
packaging==20.9
pbr==5.6.0
portalocker==1.7.1
protobuf==3.17.3
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycparser==2.20
pyparsing==2.4.7
pyrsistent==0.18.0
python-dateutil==2.8.1
pytz==2021.1
pyyaml==5.4.1
ratelimiter==1.2.0.post0
requests==2.25.1
requests-oauthlib==1.3.0
requestsexceptions==1.4.0
retrying==1.3.3
rsa==4.7.2
s3transfer==0.4.2
setuptools==56.0.0
six==1.16.0
stevedore==3.3.0
tabulate==0.8.9
typing-extensions==3.10.0.0
tzlocal==2.1
uritemplate==3.0.1
urllib3==1.26.6
websocket-client==1.1.0
zipp==3.5.0
```


### Policy

```shell
- name: sns-encrypt-transit
  resource: aws.sns
  description: All SNS topics must be encrypted in transit
  filters:
    - not:
      - type: has-statement
        statements:
          - Action:
              - sns:GetTopicAttributes
              - sns:SetTopicAttributes
              - sns:AddPermission
              - sns:RemovePermission
              - sns:DeleteTopic
              - sns:Subscribe
              - sns:ListSubscriptionsByTopic
              - sns:Publish
              - sns:Receive
            Condition:
              Bool:
                aws:SecureTransport: 'false'
            Effect: Deny
            Principal: '*'
  actions:
    - type: modify-policy
      add-statements:
      - Action:
        - sns:GetTopicAttributes
        - sns:SetTopicAttributes
        - sns:AddPermission
        - sns:RemovePermission
        - sns:DeleteTopic
        - sns:Subscribe
        - sns:ListSubscriptionsByTopic
        - sns:Publish
        - sns:Receive
        Condition:
          Bool:
            aws:SecureTransport: 'false'
        Effect: Deny
        Principal: '*'
        Resource: '*'
        Sid: DenyNotEncryptedInTrans
    - type: notify
      action_desc: 'ACTION Taken: Custodian has added the policy statement to enable encryption in transit.'
      cc:
        - custodian_team@company.com
      from: no_reply@company.com
      template: default.html
      to:
        - team@company.com
        - resource-owner
      transport:
        type: sns
        topic: arn:aws:sns:us-east-1:0123456789:mailer
      violation_desc: The following SNS Topic(s) do not have encryption in transit enabled.
```


### Relevant log/traceback output

```shell
2021-10-08 12:58:33,582 - custodian.output - ERROR - Error while executing policy
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/c7n/policy.py", line 322, in run
    results = a.process(resources)
  File "/usr/local/lib/python3.7/dist-packages/c7n/resources/sns.py", line 330, in process
    new_policy, added = self.add_statements(new_policy)
  File "/usr/local/lib/python3.7/dist-packages/c7n/actions/policy.py", line 100, in add_statements
    current = {s['Sid']: s for s in policy_statements}
  File "/usr/local/lib/python3.7/dist-packages/c7n/actions/policy.py", line 100, in 
    current = {s['Sid']: s for s in policy_statements}
KeyError: 'Sid'
```


### Extra information or context

_No response_


Discussion:

Resolving patch:
diff --git a/c7n/actions/policy.py b/c7n/actions/policy.py
--- a/c7n/actions/policy.py
+++ b/c7n/actions/policy.py
@@ -1,5 +1,6 @@
 # Copyright The Cloud Custodian Authors.
 # SPDX-License-Identifier: Apache-2.0
+import hashlib
 
 from .core import BaseAction
 from c7n import utils
@@ -41,6 +42,11 @@ def remove_statements(match_ids, statements, matched=()):
     return statements, found
 
 
+def statement_id(s):
+    # for statements without a sid, use a checksum for identity
+    return hashlib.sha224(utils.dumps(s, indent=0).encode('utf8')).hexdigest()
+
+
 class ModifyPolicyBase(BaseAction):
     """Action to modify resource IAM policy statements.
 
@@ -97,8 +103,8 @@ def __init__(self, data=None, manager=None):
         self.manager = manager
 
     def add_statements(self, policy_statements):
-        current = {s['Sid']: s for s in policy_statements}
-        additional = {s['Sid']: s for s in self.data.get('add-statements', [])}
+        current = {s.get('Sid', statement_id(s)): s for s in policy_statements}
+        additional = {s.get('Sid', statement_id(s)): s for s in self.data.get('add-statements', [])}
         current.update(additional)
         return list(current.values()), bool(additional)
 


Test patch:
diff --git a/tests/data/placebo/test_sns_modify_add_policy_without_sid/sns.CreateTopic_1.json b/tests/data/placebo/test_sns_modify_add_policy_without_sid/sns.CreateTopic_1.json
new file mode 100644
--- /dev/null
+++ b/tests/data/placebo/test_sns_modify_add_policy_without_sid/sns.CreateTopic_1.json
@@ -0,0 +1,7 @@
+{
+    "status_code": 200,
+    "data": {
+        "TopicArn": "arn:aws:sns:us-east-1:644160558196:c7n-test-rbp-no-sid",
+        "ResponseMetadata": {}
+    }
+}
\ No newline at end of file
diff --git a/tests/data/placebo/test_sns_modify_add_policy_without_sid/sns.DeleteTopic_1.json b/tests/data/placebo/test_sns_modify_add_policy_without_sid/sns.DeleteTopic_1.json
new file mode 100644
--- /dev/null
+++ b/tests/data/placebo/test_sns_modify_add_policy_without_sid/sns.DeleteTopic_1.json
@@ -0,0 +1,6 @@
+{
+    "status_code": 200,
+    "data": {
+        "ResponseMetadata": {}
+    }
+}
\ No newline at end of file
diff --git a/tests/data/placebo/test_sns_modify_add_policy_without_sid/sns.GetTopicAttributes_1.json b/tests/data/placebo/test_sns_modify_add_policy_without_sid/sns.GetTopicAttributes_1.json
new file mode 100644
--- /dev/null
+++ b/tests/data/placebo/test_sns_modify_add_policy_without_sid/sns.GetTopicAttributes_1.json
@@ -0,0 +1,16 @@
+{
+    "status_code": 200,
+    "data": {
+        "Attributes": {
+            "Policy": "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Allow\",\"Principal\":\"*\",\"Action\":\"SNS:Subscribe\",\"Resource\":\"arn:aws:sns:us-east-1:644160558196:c7n-test-rbp-no-sid\"}]}",
+            "Owner": "644160558196",
+            "SubscriptionsPending": "0",
+            "TopicArn": "arn:aws:sns:us-east-1:644160558196:c7n-test-rbp-no-sid",
+            "EffectiveDeliveryPolicy": "{\"http\":{\"defaultHealthyRetryPolicy\":{\"minDelayTarget\":20,\"maxDelayTarget\":20,\"numRetries\":3,\"numMaxDelayRetries\":0,\"numNoDelayRetries\":0,\"numMinDelayRetries\":0,\"backoffFunction\":\"linear\"},\"disableSubscriptionOverrides\":false,\"defaultRequestPolicy\":{\"headerContentType\":\"text/plain; charset=UTF-8\"}}}",
+            "SubscriptionsConfirmed": "0",
+            "DisplayName": "",
+            "SubscriptionsDeleted": "0"
+        },
+        "ResponseMetadata": {}
+    }
+}
\ No newline at end of file
diff --git a/tests/data/placebo/test_sns_modify_add_policy_without_sid/sns.GetTopicAttributes_2.json b/tests/data/placebo/test_sns_modify_add_policy_without_sid/sns.GetTopicAttributes_2.json
new file mode 100644
--- /dev/null
+++ b/tests/data/placebo/test_sns_modify_add_policy_without_sid/sns.GetTopicAttributes_2.json
@@ -0,0 +1,22 @@
+{
+    "status_code": 200,
+    "data": {
+        "Attributes": {
+            "Policy": "{\"Version\":\"2008-10-17\",\"Id\":\"__default_policy_ID\",\"Statement\":[{\"Effect\":\"Allow\",\"Principal\":{\"AWS\":\"*\"},\"Action\":[\"SNS:GetTopicAttributes\",\"SNS:SetTopicAttributes\",\"SNS:AddPermission\",\"SNS:RemovePermission\",\"SNS:DeleteTopic\",\"SNS:Subscribe\",\"SNS:ListSubscriptionsByTopic\",\"SNS:Publish\"],\"Resource\":\"arn:aws:sns:us-east-1:644160558196:test123\",\"Condition\":{\"StringEquals\":{\"AWS:SourceOwner\":\"644160558196\"}}},{\"Sid\":\"AddMe\",\"Effect\":\"Allow\",\"Principal\":\"*\",\"Action\":\"SNS:GetTopicAttributes\",\"Resource\":\"arn:aws:sns:us-east-1:644160558196:test123\"}]}",
+            "LambdaSuccessFeedbackSampleRate": "0",
+            "Owner": "644160558196",
+            "SubscriptionsPending": "0",
+            "KmsMasterKeyId": "arn:aws:kms:us-east-1:644160558196:key/082cd05f-96d1-49f6-a5ac-32093d2cfe38",
+            "TopicArn": "arn:aws:sns:us-east-1:644160558196:test123",
+            "EffectiveDeliveryPolicy": "{\"http\":{\"defaultHealthyRetryPolicy\":{\"minDelayTarget\":20,\"maxDelayTarget\":20,\"numRetries\":3,\"numMaxDelayRetries\":0,\"numNoDelayRetries\":0,\"numMinDelayRetries\":0,\"backoffFunction\":\"linear\"},\"disableSubscriptionOverrides\":false,\"defaultRequestPolicy\":{\"headerContentType\":\"text/plain; charset=UTF-8\"}}}",
+            "FirehoseSuccessFeedbackSampleRate": "0",
+            "SubscriptionsConfirmed": "0",
+            "SQSSuccessFeedbackSampleRate": "0",
+            "HTTPSuccessFeedbackSampleRate": "0",
+            "ApplicationSuccessFeedbackSampleRate": "0",
+            "DisplayName": "",
+            "SubscriptionsDeleted": "0"
+        },
+        "ResponseMetadata": {}
+    }
+}
\ No newline at end of file
diff --git a/tests/data/placebo/test_sns_modify_add_policy_without_sid/sns.GetTopicAttributes_3.json b/tests/data/placebo/test_sns_modify_add_policy_without_sid/sns.GetTopicAttributes_3.json
new file mode 100644
--- /dev/null
+++ b/tests/data/placebo/test_sns_modify_add_policy_without_sid/sns.GetTopicAttributes_3.json
@@ -0,0 +1,16 @@
+{
+    "status_code": 200,
+    "data": {
+        "Attributes": {
+            "Policy": "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Allow\",\"Principal\":\"*\",\"Action\":\"SNS:Subscribe\",\"Resource\":\"arn:aws:sns:us-east-1:644160558196:c7n-test-rbp-no-sid\"},{\"Sid\":\"AddMe\",\"Effect\":\"Allow\",\"Principal\":\"*\",\"Action\":\"SNS:GetTopicAttributes\",\"Resource\":\"arn:aws:sns:us-east-1:644160558196:c7n-test-rbp-no-sid\"}]}",
+            "Owner": "644160558196",
+            "SubscriptionsPending": "0",
+            "TopicArn": "arn:aws:sns:us-east-1:644160558196:c7n-test-rbp-no-sid",
+            "EffectiveDeliveryPolicy": "{\"http\":{\"defaultHealthyRetryPolicy\":{\"minDelayTarget\":20,\"maxDelayTarget\":20,\"numRetries\":3,\"numMaxDelayRetries\":0,\"numNoDelayRetries\":0,\"numMinDelayRetries\":0,\"backoffFunction\":\"linear\"},\"disableSubscriptionOverrides\":false,\"defaultRequestPolicy\":{\"headerContentType\":\"text/plain; charset=UTF-8\"}}}",
+            "SubscriptionsConfirmed": "0",
+            "DisplayName": "",
+            "SubscriptionsDeleted": "0"
+        },
+        "ResponseMetadata": {}
+    }
+}
\ No newline at end of file
diff --git a/tests/data/placebo/test_sns_modify_add_policy_without_sid/sns.ListTopics_1.json b/tests/data/placebo/test_sns_modify_add_policy_without_sid/sns.ListTopics_1.json
new file mode 100644
--- /dev/null
+++ b/tests/data/placebo/test_sns_modify_add_policy_without_sid/sns.ListTopics_1.json
@@ -0,0 +1,14 @@
+{
+    "status_code": 200,
+    "data": {
+        "Topics": [
+            {
+                "TopicArn": "arn:aws:sns:us-east-1:644160558196:c7n-test-rbp-no-sid"
+            },
+            {
+                "TopicArn": "arn:aws:sns:us-east-1:644160558196:test123"
+            }
+        ],
+        "ResponseMetadata": {}
+    }
+}
\ No newline at end of file
diff --git a/tests/data/placebo/test_sns_modify_add_policy_without_sid/sns.SetTopicAttributes_1.json b/tests/data/placebo/test_sns_modify_add_policy_without_sid/sns.SetTopicAttributes_1.json
new file mode 100644
--- /dev/null
+++ b/tests/data/placebo/test_sns_modify_add_policy_without_sid/sns.SetTopicAttributes_1.json
@@ -0,0 +1,6 @@
+{
+    "status_code": 200,
+    "data": {
+        "ResponseMetadata": {}
+    }
+}
\ No newline at end of file
diff --git a/tests/data/placebo/test_sns_modify_add_policy_without_sid/sns.SetTopicAttributes_2.json b/tests/data/placebo/test_sns_modify_add_policy_without_sid/sns.SetTopicAttributes_2.json
new file mode 100644
--- /dev/null
+++ b/tests/data/placebo/test_sns_modify_add_policy_without_sid/sns.SetTopicAttributes_2.json
@@ -0,0 +1,6 @@
+{
+    "status_code": 200,
+    "data": {
+        "ResponseMetadata": {}
+    }
+}
\ No newline at end of file
diff --git a/tests/data/placebo/test_sns_modify_add_policy_without_sid/tagging.GetResources_1.json b/tests/data/placebo/test_sns_modify_add_policy_without_sid/tagging.GetResources_1.json
new file mode 100644
--- /dev/null
+++ b/tests/data/placebo/test_sns_modify_add_policy_without_sid/tagging.GetResources_1.json
@@ -0,0 +1,8 @@
+{
+    "status_code": 200,
+    "data": {
+        "PaginationToken": "",
+        "ResourceTagMappingList": [],
+        "ResponseMetadata": {}
+    }
+}
\ No newline at end of file
diff --git a/tests/test_sns.py b/tests/test_sns.py
--- a/tests/test_sns.py
+++ b/tests/test_sns.py
@@ -472,6 +472,68 @@ def test_sns_modify_add_and_remove_policy(self):
         self.assertTrue("RemoveMe" not in statement_ids)
         self.assertTrue("SpecificAllow" in statement_ids)
 
+    def test_sns_modify_add_policy_without_sid(self):
+        session_factory = self.replay_flight_data("test_sns_modify_add_policy_without_sid")
+        client = session_factory().client("sns")
+        name = "c7n-test-rbp-no-sid"
+        topic_arn = client.create_topic(Name=name)["TopicArn"]
+        self.addCleanup(client.delete_topic, TopicArn=topic_arn)
+
+        client.set_topic_attributes(
+            TopicArn=topic_arn,
+            AttributeName="Policy",
+            AttributeValue=json.dumps(
+                {
+                    "Version": "2012-10-17",
+                    "Statement": [
+                        {
+                            "Effect": "Allow",
+                            "Principal": "*",
+                            "Action": ["SNS:Subscribe"],
+                            "Resource": topic_arn,
+                        }
+                    ],
+                }
+            ),
+        )
+
+        p = self.load_policy(
+            {
+                "name": "sns-modify-add-policy-without-sid",
+                "resource": "sns",
+                "filters": [{"TopicArn": topic_arn}],
+                "actions": [
+                    {
+                        "type": "modify-policy",
+                        "add-statements": [
+                            {
+                                "Sid": "AddMe",
+                                "Effect": "Allow",
+                                "Principal": "*",
+                                "Action": ["SNS:GetTopicAttributes"],
+                                "Resource": topic_arn,
+                            }
+                        ],
+                        "remove-statements": [],
+                    }
+                ],
+            },
+            session_factory=session_factory,
+        )
+
+        resources = p.run()
+        self.assertEqual(len(resources), 1)
+
+        data = json.loads(
+            client.get_topic_attributes(TopicArn=resources[0]["TopicArn"])[
+                "Attributes"
+            ][
+                "Policy"
+            ]
+        )
+        self.assertEqual(len(data.get('Statement')), 2)
+        self.assertTrue("AddMe" in [s.get("Sid") for s in data.get("Statement", ())])
+
     def test_sns_topic_encryption(self):
         session_factory = self.replay_flight_data('test_sns_kms_related_filter_test')
         kms = session_factory().client('kms', region_name='ap-northeast-2')