Repository: cloud-custodian/cloud-custodian

Issue:
S3 filters with 'or' with custodian.filters:ERROR Message: 'list' object has no attribute 'get'
**Describe the bug**
Running the policy below to check for bucket encryption with or fails with the following errors.
```
2020-10-30 10:54:20,317: custodian.filters:ERROR Message: 'list' object has no attribute 'get' Bucket: {bucket_name}
```

**To Reproduce**
Steps to reproduce the behavior:
Run ``` run --dryrun -s out s3_test.yml```

**Expected behavior**
Policy should filter buckets appropriately and filter unencrypted s3 buckets


**Background:**
 - OS: [e.g. OSX 10.15.7]
 - Python Version: [python 3.8.2]
 - Custodian Version: [0.9.7]
 - Cloud Provider: aws
 - Policy: 
```yaml
policies:
  - name: s3-test
    resource: s3
    filters:
      - or:
        - type: bucket-encryption
          state: False
        - type: bucket-encryption
          state: True
          crypto: aws:kms
          key: alias/aws/s3
        - type: bucket-encryption
          state: True
          crypto: AES256
```
 - Traceback: 
```
2020-10-30 10:54:20,887: custodian.filters:ERROR Message: 'list' object has no attribute 'get' Bucket: {bucket_name}
```
 - `custodian version --debug`
```
Custodian:   0.9.7
Python:      3.8.2 (default, Aug 25 2020, 09:23:57) 
             [Clang 12.0.0 (clang-1200.0.32.2)]
Platform:    posix.uname_result(sysname='Darwin', nodename='FVFYH1FGJK7M', release='19.6.0', version='Darwin Kernel Version 19.6.0: Mon Aug 31 22:12:52 PDT 2020; root:xnu-6153.141.2~1/RELEASE_X86_64', machine='x86_64')
Using venv:  True
Docker: False
Installed: 

argcomplete==1.12.0
attrs==19.3.0
boto3==1.14.31
botocore==1.17.31
docutils==0.15.2
importlib-metadata==1.7.0
jmespath==0.10.0
jsonpickle==1.3
jsonschema==3.2.0
pyrsistent==0.16.0
python-dateutil==2.8.1
pyyaml==5.3.1
s3transfer==0.3.0
setuptools==41.2.0
six==1.14.0
tabulate==0.8.7
urllib3==1.24.3
zipp==3.1.0

```

**Additional context**
Tested this policy in 0.8.33.0 and it worked as expected
This issue is related to #6083 found on 0.9.5



Discussion:
@kapilt , I would appreciate your feedback on this. Thank you
Looks like the issue still persist in 0.9.7
Any updates on this issue?

Resolving patch:
diff --git a/c7n/resources/s3.py b/c7n/resources/s3.py
--- a/c7n/resources/s3.py
+++ b/c7n/resources/s3.py
@@ -3215,7 +3215,7 @@ def process_bucket(self, b):
                 be = {}
             b[self.annotation_key] = be
         else:
-            be = [self.annotation_key]
+            be = b[self.annotation_key]
 
         rules = be.get('ServerSideEncryptionConfiguration', {}).get('Rules', [])
         # default `state` to True as previous impl assumed state == True


Test patch:
diff --git a/tests/data/placebo/test_s3_encryption_audit/kms.DescribeKey_1.json b/tests/data/placebo/test_s3_encryption_audit/kms.DescribeKey_1.json
new file mode 100644
--- /dev/null
+++ b/tests/data/placebo/test_s3_encryption_audit/kms.DescribeKey_1.json
@@ -0,0 +1,31 @@
+{
+    "status_code": 200,
+    "data": {
+        "KeyMetadata": {
+            "AWSAccountId": "644160558196",
+            "KeyId": "51e07389-e68f-4713-aa2e-c913565057b9",
+            "Arn": "arn:aws:kms:us-east-1:644160558196:key/51e07389-e68f-4713-aa2e-c913565057b9",
+            "CreationDate": {
+                "__class__": "datetime",
+                "year": 2021,
+                "month": 2,
+                "day": 10,
+                "hour": 3,
+                "minute": 54,
+                "second": 50,
+                "microsecond": 599000
+            },
+            "Enabled": true,
+            "Description": "Default master key that protects my S3 objects when no other key is defined",
+            "KeyUsage": "ENCRYPT_DECRYPT",
+            "KeyState": "Enabled",
+            "Origin": "AWS_KMS",
+            "KeyManager": "AWS",
+            "CustomerMasterKeySpec": "SYMMETRIC_DEFAULT",
+            "EncryptionAlgorithms": [
+                "SYMMETRIC_DEFAULT"
+            ]
+        },
+        "ResponseMetadata": {}
+    }
+}
\ No newline at end of file
diff --git a/tests/data/placebo/test_s3_encryption_audit/s3.GetBucketEncryption_1.json b/tests/data/placebo/test_s3_encryption_audit/s3.GetBucketEncryption_1.json
new file mode 100644
--- /dev/null
+++ b/tests/data/placebo/test_s3_encryption_audit/s3.GetBucketEncryption_1.json
@@ -0,0 +1,16 @@
+{
+    "status_code": 200,
+    "data": {
+        "ResponseMetadata": {},
+        "ServerSideEncryptionConfiguration": {
+            "Rules": [
+                {
+                    "ApplyServerSideEncryptionByDefault": {
+                        "SSEAlgorithm": "aws:kms"
+                    },
+                    "BucketKeyEnabled": false
+                }
+            ]
+        }
+    }
+}
\ No newline at end of file
diff --git a/tests/data/placebo/test_s3_encryption_audit/s3.GetBucketEncryption_2.json b/tests/data/placebo/test_s3_encryption_audit/s3.GetBucketEncryption_2.json
new file mode 100644
--- /dev/null
+++ b/tests/data/placebo/test_s3_encryption_audit/s3.GetBucketEncryption_2.json
@@ -0,0 +1,16 @@
+{
+    "status_code": 200,
+    "data": {
+        "ResponseMetadata": {},
+        "ServerSideEncryptionConfiguration": {
+            "Rules": [
+                {
+                    "ApplyServerSideEncryptionByDefault": {
+                        "SSEAlgorithm": "AES256"
+                    },
+                    "BucketKeyEnabled": false
+                }
+            ]
+        }
+    }
+}
\ No newline at end of file
diff --git a/tests/data/placebo/test_s3_encryption_audit/s3.GetBucketEncryption_3.json b/tests/data/placebo/test_s3_encryption_audit/s3.GetBucketEncryption_3.json
new file mode 100644
--- /dev/null
+++ b/tests/data/placebo/test_s3_encryption_audit/s3.GetBucketEncryption_3.json
@@ -0,0 +1,11 @@
+{
+    "status_code": 404,
+    "data": {
+        "Error": {
+            "Code": "ServerSideEncryptionConfigurationNotFoundError",
+            "Message": "The server side encryption configuration was not found",
+            "BucketName": "c7n-aws-s3-encryption-audit-test-c"
+        },
+        "ResponseMetadata": {}
+    }
+}
\ No newline at end of file
diff --git a/tests/data/placebo/test_s3_encryption_audit/s3.ListBuckets_1.json b/tests/data/placebo/test_s3_encryption_audit/s3.ListBuckets_1.json
new file mode 100644
--- /dev/null
+++ b/tests/data/placebo/test_s3_encryption_audit/s3.ListBuckets_1.json
@@ -0,0 +1,51 @@
+{
+    "status_code": 200,
+    "data": {
+        "ResponseMetadata": {},
+        "Buckets": [
+            {
+                "Name": "c7n-aws-s3-encryption-audit-test-a",
+                "CreationDate": {
+                    "__class__": "datetime",
+                    "year": 2021,
+                    "month": 2,
+                    "day": 12,
+                    "hour": 13,
+                    "minute": 48,
+                    "second": 0,
+                    "microsecond": 0
+                }
+            },
+            {
+                "Name": "c7n-aws-s3-encryption-audit-test-b",
+                "CreationDate": {
+                    "__class__": "datetime",
+                    "year": 2021,
+                    "month": 2,
+                    "day": 12,
+                    "hour": 13,
+                    "minute": 48,
+                    "second": 0,
+                    "microsecond": 0
+                }
+            },
+            {
+                "Name": "c7n-aws-s3-encryption-audit-test-c",
+                "CreationDate": {
+                    "__class__": "datetime",
+                    "year": 2021,
+                    "month": 2,
+                    "day": 12,
+                    "hour": 13,
+                    "minute": 48,
+                    "second": 0,
+                    "microsecond": 0
+                }
+            }
+        ],
+        "Owner": {
+            "DisplayName": "wayne",
+            "ID": "ed8f0ad562d627dcda7d11c0f27748b03f2c87ba6ab69cbd6e1acd390e5f0047"
+        }
+    }
+}
\ No newline at end of file
diff --git a/tests/terraform/aws_s3_encryption_audit/main.tf b/tests/terraform/aws_s3_encryption_audit/main.tf
new file mode 100644
--- /dev/null
+++ b/tests/terraform/aws_s3_encryption_audit/main.tf
@@ -0,0 +1,28 @@
+resource "aws_s3_bucket" "example_a" {
+  bucket = "c7n-aws-s3-encryption-audit-test-a"
+  acl    = "private"
+  server_side_encryption_configuration {
+    rule {
+      apply_server_side_encryption_by_default {
+        sse_algorithm = "aws:kms"
+      }
+    }
+  }
+}
+
+resource "aws_s3_bucket" "example_b" {
+  bucket = "c7n-aws-s3-encryption-audit-test-b"
+  acl    = "private"
+  server_side_encryption_configuration {
+    rule {
+      apply_server_side_encryption_by_default {
+        sse_algorithm = "AES256"
+      }
+    }
+  }
+}
+
+resource "aws_s3_bucket" "example_c" {
+  bucket = "c7n-aws-s3-encryption-audit-test-c"
+  acl    = "private"
+}
diff --git a/tests/terraform/aws_s3_encryption_audit/tf_resources.json b/tests/terraform/aws_s3_encryption_audit/tf_resources.json
new file mode 100644
--- /dev/null
+++ b/tests/terraform/aws_s3_encryption_audit/tf_resources.json
@@ -0,0 +1,130 @@
+{
+    "pytest-terraform": 1,
+    "outputs": {},
+    "resources": {
+        "aws_s3_bucket": {
+            "example_a": {
+                "acceleration_status": "",
+                "acl": "private",
+                "arn": "arn:aws:s3:::c7n-aws-s3-encryption-audit-test-a",
+                "bucket": "c7n-aws-s3-encryption-audit-test-a",
+                "bucket_domain_name": "c7n-aws-s3-encryption-audit-test-a.s3.amazonaws.com",
+                "bucket_prefix": null,
+                "bucket_regional_domain_name": "c7n-aws-s3-encryption-audit-test-a.s3.amazonaws.com",
+                "cors_rule": [],
+                "force_destroy": false,
+                "grant": [],
+                "hosted_zone_id": "Z3AQBSTGFYJSTF",
+                "id": "c7n-aws-s3-encryption-audit-test-a",
+                "lifecycle_rule": [],
+                "logging": [],
+                "object_lock_configuration": [],
+                "policy": null,
+                "region": "us-east-1",
+                "replication_configuration": [],
+                "request_payer": "BucketOwner",
+                "server_side_encryption_configuration": [
+                    {
+                        "rule": [
+                            {
+                                "apply_server_side_encryption_by_default": [
+                                    {
+                                        "kms_master_key_id": "",
+                                        "sse_algorithm": "aws:kms"
+                                    }
+                                ]
+                            }
+                        ]
+                    }
+                ],
+                "tags": null,
+                "versioning": [
+                    {
+                        "enabled": false,
+                        "mfa_delete": false
+                    }
+                ],
+                "website": [],
+                "website_domain": null,
+                "website_endpoint": null
+            },
+            "example_b": {
+                "acceleration_status": "",
+                "acl": "private",
+                "arn": "arn:aws:s3:::c7n-aws-s3-encryption-audit-test-b",
+                "bucket": "c7n-aws-s3-encryption-audit-test-b",
+                "bucket_domain_name": "c7n-aws-s3-encryption-audit-test-b.s3.amazonaws.com",
+                "bucket_prefix": null,
+                "bucket_regional_domain_name": "c7n-aws-s3-encryption-audit-test-b.s3.amazonaws.com",
+                "cors_rule": [],
+                "force_destroy": false,
+                "grant": [],
+                "hosted_zone_id": "Z3AQBSTGFYJSTF",
+                "id": "c7n-aws-s3-encryption-audit-test-b",
+                "lifecycle_rule": [],
+                "logging": [],
+                "object_lock_configuration": [],
+                "policy": null,
+                "region": "us-east-1",
+                "replication_configuration": [],
+                "request_payer": "BucketOwner",
+                "server_side_encryption_configuration": [
+                    {
+                        "rule": [
+                            {
+                                "apply_server_side_encryption_by_default": [
+                                    {
+                                        "kms_master_key_id": "",
+                                        "sse_algorithm": "AES256"
+                                    }
+                                ]
+                            }
+                        ]
+                    }
+                ],
+                "tags": null,
+                "versioning": [
+                    {
+                        "enabled": false,
+                        "mfa_delete": false
+                    }
+                ],
+                "website": [],
+                "website_domain": null,
+                "website_endpoint": null
+            },
+            "example_c": {
+                "acceleration_status": "",
+                "acl": "private",
+                "arn": "arn:aws:s3:::c7n-aws-s3-encryption-audit-test-c",
+                "bucket": "c7n-aws-s3-encryption-audit-test-c",
+                "bucket_domain_name": "c7n-aws-s3-encryption-audit-test-c.s3.amazonaws.com",
+                "bucket_prefix": null,
+                "bucket_regional_domain_name": "c7n-aws-s3-encryption-audit-test-c.s3.amazonaws.com",
+                "cors_rule": [],
+                "force_destroy": false,
+                "grant": [],
+                "hosted_zone_id": "Z3AQBSTGFYJSTF",
+                "id": "c7n-aws-s3-encryption-audit-test-c",
+                "lifecycle_rule": [],
+                "logging": [],
+                "object_lock_configuration": [],
+                "policy": null,
+                "region": "us-east-1",
+                "replication_configuration": [],
+                "request_payer": "BucketOwner",
+                "server_side_encryption_configuration": [],
+                "tags": null,
+                "versioning": [
+                    {
+                        "enabled": false,
+                        "mfa_delete": false
+                    }
+                ],
+                "website": [],
+                "website_domain": null,
+                "website_endpoint": null
+            }
+        }
+    }
+}
\ No newline at end of file
diff --git a/tests/test_s3.py b/tests/test_s3.py
--- a/tests/test_s3.py
+++ b/tests/test_s3.py
@@ -3610,3 +3610,52 @@ def run_policy(policy):
         lifecycle = client.get_bucket_lifecycle_configuration(Bucket=bname)
         self.assertEqual(len(lifecycle["Rules"]), 1)
         self.assertEqual(lifecycle["Rules"][0]["ID"], lifecycle_id2)
+
+
+@terraform('aws_s3_encryption_audit')
+def test_s3_encryption_audit(test, aws_s3_encryption_audit):
+    test.patch(s3.S3, "executor_factory", MainThreadExecutor)
+    test.patch(s3.BucketEncryption, "executor_factory", MainThreadExecutor)
+    test.patch(s3, "S3_AUGMENT_TABLE", [])
+    session_factory = test.replay_flight_data("test_s3_encryption_audit")
+
+    p = test.load_policy(
+        {
+            "name": "s3-audit",
+            "resource": "s3",
+            "filters": [
+                {
+                    "or": [
+                        {
+                            "type": "bucket-encryption",
+                            "state": False,
+                        },
+                        {
+                            "type": "bucket-encryption",
+                            "crypto": "aws:kms",
+                            "state": True,
+                        },
+                        {
+                            "type": "bucket-encryption",
+                            "crypto": "AES256",
+                            "state": True,
+                        },
+                    ]
+                },
+            ],
+        },
+        session_factory=session_factory,
+    )
+
+    resources = p.run()
+
+    assert len(resources) == 3
+
+    expected_names = [
+        'c7n-aws-s3-encryption-audit-test-a',
+        'c7n-aws-s3-encryption-audit-test-b',
+        'c7n-aws-s3-encryption-audit-test-c',
+    ]
+    actual_names = sorted([r.get('Name') for r in resources])
+
+    assert actual_names == expected_names