Repository: cloud-custodian/cloud-custodian

Issue:
s3 filter check-public-block causes c7n to fail to scan remaining buckets when it encounters an access denied
**Describe the bug**
When applying the s3 filter 'check-public-block' c7n fails when it encounters one bucket with "Access Denied" and does not continue to scan the remaining buckets

**To Reproduce**
Create a bucket with a bucket policy that denies access to your c7n role
Create a bucket with block public access disabled
Run a c7n policy that contains the following filter

```
policies:
  - name: check-public-block
    resource: s3
    filters:
      - type: check-public-block
```

**Expected behavior**
I expect c7n to log the Access Denied error for the restricted bucket, then continue to scan the remaining buckets in the account and flag the bucket with block public access disabled.


**Background (please complete the following information):**
 - OS: [e.g. OSX 10.15] macOS 10.15.7
 - Python Version: [e.g. python 3.8.1] Python 3.9.4
 - Custodian Version: [e.g. 0.8.46.1] 0.9.11
 - Tool Version: [if applicable]
 - Cloud Provider: [e.g. gcp, aws, azure] AWS
 - Policy: [please exclude any account/sensitive information]
```yaml
policies:
  - name: check-public-block
    resource: s3
    filters:
      - type: check-public-block
```
 - Traceback: [if applicable, please exclude sensitive/account information]

2021-05-13 12:20:38,102: custodian.s3:WARNING Bucket:my-restricted-bucket unable to invoke method:get_bucket_location error:Access Denied 
2021-05-13 12:20:38,433: custodian.s3:WARNING Bucket:my-restricted-bucket unable to invoke method:get_bucket_policy error:Access Denied 
2021-05-13 12:20:38,484: custodian.s3:WARNING Bucket:my-restricted-bucket unable to invoke method:get_bucket_acl error:Access Denied 
2021-05-13 12:20:38,535: custodian.s3:WARNING Bucket:my-restricted-bucket unable to invoke method:get_bucket_replication error:Access Denied 
2021-05-13 12:20:38,581: custodian.s3:WARNING Bucket:my-restricted-bucket unable to invoke method:get_bucket_versioning error:Access Denied 
2021-05-13 12:20:38,626: custodian.s3:WARNING Bucket:my-restricted-bucket unable to invoke method:get_bucket_website error:Access Denied 
2021-05-13 12:20:38,670: custodian.s3:WARNING Bucket:my-restricted-bucket unable to invoke method:get_bucket_logging error:Access Denied 
2021-05-13 12:20:38,714: custodian.s3:WARNING Bucket:my-restricted-bucket unable to invoke method:get_bucket_notification_configuration error:Access Denied 
2021-05-13 12:20:38,855: custodian.s3:WARNING Bucket:my-restricted-bucket unable to invoke method:get_bucket_lifecycle_configuration error:Access Denied 
2021-05-13 12:20:44,003: custodian.output:ERROR Error while executing policy
Traceback (most recent call last):
  File "/Users/user/github.com/cloud-custodian/cloud-custodian/custodian/lib/python3.9/site-packages/c7n/policy.py", line 285, in run
    resources = self.policy.resource_manager.resources()
  File "/Users/user/github.com/cloud-custodian/cloud-custodian/custodian/lib/python3.9/site-packages/c7n/query.py", line 523, in resources
    resources = self.filter_resources(resources)
  File "/Users/user/github.com/cloud-custodian/cloud-custodian/custodian/lib/python3.9/site-packages/c7n/manager.py", line 111, in filter_resources
    resources = f.process(resources, event)
  File "/Users/user/github.com/cloud-custodian/cloud-custodian/custodian/lib/python3.9/site-packages/c7n/resources/s3.py", line 1415, in process
    if f.result():
  File "/usr/local/Cellar/python@3.9/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/concurrent/futures/_base.py", line 438, in result
    return self.__get_result()
  File "/usr/local/Cellar/python@3.9/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/concurrent/futures/_base.py", line 390, in __get_result
    raise self._exception
  File "/usr/local/Cellar/python@3.9/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/concurrent/futures/thread.py", line 52, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/user/github.com/cloud-custodian/cloud-custodian/custodian/lib/python3.9/site-packages/c7n/resources/s3.py", line 1424, in process_bucket
    config = s3.get_public_access_block(
  File "/Users/user/github.com/cloud-custodian/cloud-custodian/custodian/lib/python3.9/site-packages/botocore/client.py", line 357, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/Users/user/github.com/cloud-custodian/cloud-custodian/custodian/lib/python3.9/site-packages/botocore/client.py", line 676, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the GetPublicAccessBlock operation: Access Denied
Traceback (most recent call last):
  File "/Users/user/github.com/cloud-custodian/cloud-custodian/custodian/lib/python3.9/site-packages/c7n/cli.py", line 352, in main
    command(config)
  File "/Users/user/github.com/cloud-custodian/cloud-custodian/custodian/lib/python3.9/site-packages/c7n/commands.py", line 125, in _load_policies
    return f(options, list(policies))
  File "/Users/user/github.com/cloud-custodian/cloud-custodian/custodian/lib/python3.9/site-packages/c7n/commands.py", line 271, in run
    policy()
  File "/Users/user/github.com/cloud-custodian/cloud-custodian/custodian/lib/python3.9/site-packages/c7n/policy.py", line 1178, in __call__
    resources = PullMode(self).run()
  File "/Users/user/github.com/cloud-custodian/cloud-custodian/custodian/lib/python3.9/site-packages/c7n/policy.py", line 285, in run
    resources = self.policy.resource_manager.resources()
  File "/Users/user/github.com/cloud-custodian/cloud-custodian/custodian/lib/python3.9/site-packages/c7n/query.py", line 523, in resources
    resources = self.filter_resources(resources)
  File "/Users/user/github.com/cloud-custodian/cloud-custodian/custodian/lib/python3.9/site-packages/c7n/manager.py", line 111, in filter_resources
    resources = f.process(resources, event)
  File "/Users/user/github.com/cloud-custodian/cloud-custodian/custodian/lib/python3.9/site-packages/c7n/resources/s3.py", line 1415, in process
    if f.result():
  File "/usr/local/Cellar/python@3.9/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/concurrent/futures/_base.py", line 438, in result
    return self.__get_result()
  File "/usr/local/Cellar/python@3.9/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/concurrent/futures/_base.py", line 390, in __get_result
    raise self._exception
  File "/usr/local/Cellar/python@3.9/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/concurrent/futures/thread.py", line 52, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/user/github.com/cloud-custodian/cloud-custodian/custodian/lib/python3.9/site-packages/c7n/resources/s3.py", line 1424, in process_bucket
    config = s3.get_public_access_block(
  File "/Users/user/github.com/cloud-custodian/cloud-custodian/custodian/lib/python3.9/site-packages/botocore/client.py", line 357, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/Users/user/github.com/cloud-custodian/cloud-custodian/custodian/lib/python3.9/site-packages/botocore/client.py", line 676, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the GetPublicAccessBlock operation: Access Denied
> /Users/user/github.com/cloud-custodian/cloud-custodian/custodian/lib/python3.9/site-packages/botocore/client.py(676)_make_api_call()
-> raise error_class(parsed_response, operation_name)

 - `custodian version --debug` output

Please copy/paste the following info along with any bug reports:

Custodian:   0.9.11
Python:      3.9.4 (default, Apr  5 2021, 01:49:30) 
             [Clang 12.0.0 (clang-1200.0.32.29)]
Platform:    posix.uname_result(sysname='Darwin', nodename='my-laptop', release='19.6.0', version='Darwin Kernel Version 19.6.0: Mon Apr 12 20:57:45 PDT 2021; root:xnu-6153.141.28.1~1/RELEASE_X86_64', machine='x86_64')
Using venv:  True
Docker: False
Installed: 

argcomplete==1.12.2
attrs==20.3.0
boto3==1.17.33
botocore==1.20.33
importlib-metadata==3.7.3
jmespath==0.10.0
jsonpickle==1.3
jsonschema==3.2.0
pyrsistent==0.17.3
python-dateutil==2.8.1
pyyaml==5.4.1
s3transfer==0.3.6
setuptools==54.2.0
six==1.15.0
tabulate==0.8.9
typing-extensions==3.7.4.3
urllib3==1.26.4
zipp==3.4.1

**Additional context**
Add any other context about the problem here.



Discussion:
One of the reasons could be - an ACL or policy on that specific bucket is not allowing your custodian role to scan it. You must check-in all other accounts and other S3 buckets- if you are getting the same ACCESS DENIED for all.
Correct, we have bucket policy that denies c7n access to the bucket. The issue is, c7n does not gracefully fail on the 1 bucket and continue to scan the others. It crashes and fails to scan the other buckets in the AWS account, which are accessible.
Hey guys,
Even I was getting similar issue which said 
 Access denied api:GetPublicAccessBlock policy:CheckForPublicAclBlock-Off account:AWS_IT_SANDBOX region:us-east-1
 c7n_org:INFO Policy resource counts Counter()
I tried adding below IAM policy  s3:GetBucketPublicAccessBlock 
But still I get same error
Any suggestion ?
I just encountered this today as well, and it appears there are similar issues with other S3 bucket filters like 'ownership' and 'bucket-encryption'.  There is code that gracefully handles AccessDenied for some of the bucket augments, but not these filters.

I consider this to be a fairly major issue because we have developers that sometimes mistakenly get a little aggressive in their bucket Deny policies and, just one of those in an account completely breaks these policy executions.

Resolving patch:
diff --git a/c7n/resources/s3.py b/c7n/resources/s3.py
--- a/c7n/resources/s3.py
+++ b/c7n/resources/s3.py
@@ -1370,7 +1370,19 @@ def process_bucket(self, bucket):
                 config = s3.get_public_access_block(
                     Bucket=bucket['Name'])['PublicAccessBlockConfiguration']
             except ClientError as e:
-                if e.response['Error']['Code'] != 'NoSuchPublicAccessBlockConfiguration':
+                error_code = e.response['Error']['Code']
+                if error_code == 'NoSuchPublicAccessBlockConfiguration':
+                    pass
+                elif error_code == 'AccessDenied':
+                    # Follow the same logic as `assemble_bucket` - log and continue on access
+                    # denied errors rather than halting a policy altogether
+                    method = 'GetPublicAccessBlock'
+                    log.warning(
+                        "Bucket:%s unable to invoke method:%s error:%s ",
+                        bucket['Name'], method, e.response['Error']['Message']
+                    )
+                    bucket.setdefault('c7n:DeniedMethods', []).append(method)
+                else:
                     raise
             bucket[self.annotation_key] = config
         return self.matches_filter(config)


Test patch:
diff --git a/tests/data/placebo/test_s3_check_public_block/s3.GetPublicAccessBlock_1.json b/tests/data/placebo/test_s3_check_public_block/s3.GetPublicAccessBlock_1.json
new file mode 100644
--- /dev/null
+++ b/tests/data/placebo/test_s3_check_public_block/s3.GetPublicAccessBlock_1.json
@@ -0,0 +1,11 @@
+{
+    "status_code": 404,
+    "data": {
+        "Error": {
+            "Code": "NoSuchPublicAccessBlockConfiguration",
+            "Message": "The public access block configuration was not found",
+            "BucketName": "644160558196-us-east-1-s3-logs"
+        },
+        "ResponseMetadata": {}
+    }
+}
\ No newline at end of file
diff --git a/tests/data/placebo/test_s3_check_public_block/s3.GetPublicAccessBlock_2.json b/tests/data/placebo/test_s3_check_public_block/s3.GetPublicAccessBlock_2.json
new file mode 100644
--- /dev/null
+++ b/tests/data/placebo/test_s3_check_public_block/s3.GetPublicAccessBlock_2.json
@@ -0,0 +1,12 @@
+{
+    "status_code": 200,
+    "data": {
+        "ResponseMetadata": {},
+        "PublicAccessBlockConfiguration": {
+            "BlockPublicAcls": true,
+            "IgnorePublicAcls": false,
+            "BlockPublicPolicy": true,
+            "RestrictPublicBuckets": true
+        }
+    }
+}
diff --git a/tests/data/placebo/test_s3_check_public_block/s3.GetPublicAccessBlock_3.json b/tests/data/placebo/test_s3_check_public_block/s3.GetPublicAccessBlock_3.json
new file mode 100644
--- /dev/null
+++ b/tests/data/placebo/test_s3_check_public_block/s3.GetPublicAccessBlock_3.json
@@ -0,0 +1,10 @@
+{
+    "status_code": 403,
+    "data": {
+        "Error": {
+            "Code": "AccessDenied",
+            "Message": "Access Denied"
+        },
+        "ResponseMetadata": {}
+    }
+}
\ No newline at end of file
diff --git a/tests/data/placebo/test_s3_check_public_block/s3.ListBuckets_1.json b/tests/data/placebo/test_s3_check_public_block/s3.ListBuckets_1.json
new file mode 100644
--- /dev/null
+++ b/tests/data/placebo/test_s3_check_public_block/s3.ListBuckets_1.json
@@ -0,0 +1,51 @@
+{
+    "status_code": 200,
+    "data": {
+        "ResponseMetadata": {},
+        "Buckets": [
+            {
+                "Name": "644160558196-us-east-1-s3-logs",
+                "CreationDate": {
+                    "__class__": "datetime",
+                    "year": 2021,
+                    "month": 10,
+                    "day": 19,
+                    "hour": 18,
+                    "minute": 6,
+                    "second": 8,
+                    "microsecond": 0
+                }
+            },
+            {
+                "Name": "644160558196-us-east-2-s3-logs",
+                "CreationDate": {
+                    "__class__": "datetime",
+                    "year": 2021,
+                    "month": 10,
+                    "day": 19,
+                    "hour": 15,
+                    "minute": 22,
+                    "second": 50,
+                    "microsecond": 0
+                }
+            },
+            {
+                "Name": "my-locked-down-bucket",
+                "CreationDate": {
+                    "__class__": "datetime",
+                    "year": 2022,
+                    "month": 8,
+                    "day": 31,
+                    "hour": 12,
+                    "minute": 35,
+                    "second": 48,
+                    "microsecond": 0
+                }
+            }
+        ],
+        "Owner": {
+            "DisplayName": "aws-backoffice+sandbox-aj",
+            "ID": "e46bca81657fdc9f58bc5bab457564bb66f16636bd724a9f51d212fa2185a2b0"
+        }
+    }
+}
diff --git a/tests/test_s3.py b/tests/test_s3.py
--- a/tests/test_s3.py
+++ b/tests/test_s3.py
@@ -1789,6 +1789,39 @@ def test_bucket_replication_policy_disable(self):
         for rule in response['ReplicationConfiguration']['Rules']:
             self.assertEqual(rule['Status'], 'Disabled')
 
+    def test_check_public_block(self):
+        """Handle cases where public block details are missing or unavailable
+
+        The default check-public-block filter should match buckets
+        in any of the following conditions:
+
+        - There is a public block configuration, but some settings are not
+          enabled
+        - There is no public block configuration set
+        - A strict bucket policy prevents Custodian from reading the public block configuration
+        """
+        self.patch(s3.FilterPublicBlock, "executor_factory", MainThreadExecutor)
+        self.patch(s3, "S3_AUGMENT_TABLE", [])
+
+        session_factory = self.replay_flight_data("test_s3_check_public_block")
+        p = self.load_policy(
+            {
+                "name": "check-public-block",
+                "resource": "s3",
+                "filters": [
+                    {
+                        "type": "check-public-block",
+                    }
+                ],
+            },
+            session_factory=session_factory,
+        )
+
+        resources = {bucket["Name"]: bucket for bucket in p.run()}
+        self.assertEqual(len(resources), 3)
+        locked_down_bucket = resources["my-locked-down-bucket"]
+        self.assertIn("GetPublicAccessBlock", locked_down_bucket["c7n:DeniedMethods"])
+
     def test_set_public_block_enable_all(self):
         bname = 'mypublicblock'