Repository: cloud-custodian/cloud-custodian

Issue:
Organization trails break the data-events filter for the s3 resource
### Describe the bug

I would like to find all S3 buckets that have CloudTrail data-events enabled.

It looks like custodian fails when it sees that I have an Organization Trail (i.e. non-local) enabled across my accounts.

Details below

### What did you expect to happen?

I would like Custodian to show me the local S3 buckets that have data-events enabled.

I _think_ the fix would be to filter our non-local CloudTrail trails but I am not sure.

### Cloud Provider

Amazon Web Services (AWS)

### Cloud Custodian version and dependency information

```shell
Custodian:   0.9.31
Python:      3.8.16 (default, Mar 10 2023, 03:07:48) 
             [GCC 7.3.1 20180712 (Red Hat 7.3.1-15)]
Platform:    posix.uname_result(sysname='Linux', nodename='c7ndevbox', release='4.14.320-243.544.amzn2.x86_64', version='#1 SMP Tue Aug 1 21:03:08 UTC 2023', machine='x86_64')
Using venv:  False
Docker: False
Installed: 

argcomplete==3.1.1
attrs==23.1.0
boto3==1.28.41
botocore==1.31.41
docutils==0.18.1
importlib-metadata==5.2.0
importlib-resources==6.0.1
jmespath==1.0.1
jsonschema==4.19.0
jsonschema-specifications==2023.7.1
pkgutil-resolve-name==1.3.10
python-dateutil==2.8.2
pyyaml==6.0.1
referencing==0.30.2
rpds-py==0.10.2
s3transfer==0.6.2
six==1.16.0
tabulate==0.9.0
urllib3==1.26.16
zipp==3.16.2
```


### Policy

```shell
- name: s3-bucket-report-data-events-enabled
    description: |
      Report S3 Buckets that have data events enabled
    resource: s3
    conditions:
      - region: us-east-1
    filters:
      - data-events
```


### Relevant log/traceback output

```shell
c7n-org logfile:
2023-09-14 18:19:04,092: c7n_org:DEBUG Running policy:s3-bucket-report-data-events-enabled account:ghx-rd region:us-east-1
2023-09-14 18:19:11,536: c7n_org:ERROR Exception running policy:s3-bucket-report-data-events-enabled account:ghx-rd region:us-east-1 error:An error occurred (TrailNotFoundException) when calling the GetEventSelectors operation: Unknown trail: arn:aws:cloudtrail:us-east-1:XXX:trail/OrganizationTrail for the user: XXX
2023-09-14 18:19:11,555: c7n_org:INFO Policy resource counts Counter()
```


### Extra information or context

XXX is the account number for the local account but the correct arn for OrganizationTrail includes the account number for the payer account that hosts the OrganizationTrail.


Discussion:
@ajkerrigan @kapilt Please advise me if i'm missing anything.
Even I tried with ignoring the remote account id's the code not working facing issues

```
    def get_event_buckets(self, session, trails):
        """Return a mapping of bucket name to cloudtrail.

        For wildcard trails the bucket name is ''.
        """
        regions = {t.get('HomeRegion') for t in trails}
        clients = {}
        for region in regions:
            clients[region] = session.client('cloudtrail', region_name=region)

        event_buckets = {}
        for t in trails:
            self.log.error('%s %s', 'events', clients[t.get('HomeRegion')].get_event_selectors(TrailName=t['Name']))
            for events in clients[t.get('HomeRegion')].get_event_selectors(
                    TrailName=t['Name']).get('EventSelectors', ()):
                self.log.error('%s %s', 'events', events)
                if 'DataResources' not in events:
                    continue
                for data_events in events['DataResources']:
                    if data_events['Type'] != 'AWS::S3::Object':
                        continue
                    for b in data_events['Values']:
                        event_buckets[b.rsplit(':')[-1].strip('/')] = t['Name']
        return event_buckets
        
    def process(self, resources, event=None):
        tot_trails = self.manager.get_resource_manager('cloudtrail').resources()
        account_id = self.manager.config.account_id
        trails = [i for i in tot_trails if account_id in i['TrailARN']]
        session = local_session(self.manager.session_factory)
        event_buckets = self.get_event_buckets(session, trails)
        ops = {
            'present': lambda x: (
                x['Name'] in event_buckets or '' in event_buckets),
            'absent': (
                lambda x: x['Name'] not in event_buckets and ''
                not in event_buckets)}
        op = ops[self.data['state']]
        results = []
        for b in resources:
            if op(b):
                self.log.debug('%s %s', 'buckets', op(b))
                results.append(b)
        self.log.debug('%s %s', 'results', results)
        return results
```

        
        Faced the below:
2023-09-21 13:59:51,119: c7n_org:DEBUG Running policy:s3-bucket-report-data-events-enabled account:rd region:us-east-1
2023-09-21 14:00:04,188: custodian.filters:DEBUG events {'TrailARN': 'arn:aws:cloudtrail:us-east-1:acct_id:trail/c7ndemo', 'AdvancedEventSelectors': [{'FieldSelectors': [{'Field': 'eventCategory', 'Equals': ['Data']}, {'Field': 'resources.type', 'Equals': ['AWS::S3::Object']}]}], 'ResponseMetadata': {'RequestId': '5180d81a-3d80-4313-81a2-104da02e89b0', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '5180d81a-3d80-4313-81a2-104da02e89b0', 'content-type': 'application/x-amz-json-1.1', 'content-length': '218', 'date': 'Thu, 21 Sep 2023 14:00:03 GMT'}, 'RetryAttempts': 0}}
2023-09-21 14:00:04,497: custodian.filters:INFO results []
2023-09-21 14:00:04,518: c7n_org:INFO Policy resource counts Counter({'s3-bucket-report-data-events-enabled': 0})

Actually its not try to get the below value:

```
clients[t.get('HomeRegion')].get_event_selectors(TrailName=t['Name']).get('EventSelectors', ()
```

Apart from this further proceed means:
there is no value for events['DataResources'] whether this worked earlier? Its strange or from AWS side any changes happened in the outputs?

Resolving patch:
diff --git a/c7n/resources/s3.py b/c7n/resources/s3.py
--- a/c7n/resources/s3.py
+++ b/c7n/resources/s3.py
@@ -2564,6 +2564,11 @@ def process_resource_set(self, client, resource_set, tags):
 
 @filters.register('data-events')
 class DataEvents(Filter):
+    """Find buckets for which CloudTrail is logging data events.
+
+    Note that this filter only examines trails that are defined in the
+    current account.
+    """
 
     schema = type_schema('data-events', state={'enum': ['present', 'absent']})
     permissions = (
@@ -2595,8 +2600,12 @@ def get_event_buckets(self, session, trails):
 
     def process(self, resources, event=None):
         trails = self.manager.get_resource_manager('cloudtrail').resources()
+        local_trails = self.filter_resources(
+            trails,
+            "split(':', TrailARN)[4]", (self.manager.account_id,)
+        )
         session = local_session(self.manager.session_factory)
-        event_buckets = self.get_event_buckets(session, trails)
+        event_buckets = self.get_event_buckets(session, local_trails)
         ops = {
             'present': lambda x: (
                 x['Name'] in event_buckets or '' in event_buckets),
@@ -2604,7 +2613,7 @@ def process(self, resources, event=None):
                 lambda x: x['Name'] not in event_buckets and ''
                 not in event_buckets)}
 
-        op = ops[self.data['state']]
+        op = ops[self.data.get('state', 'present')]
         results = []
         for b in resources:
             if op(b):


Test patch:
diff --git a/tests/data/placebo/test_s3_data_events/cloudtrail.DescribeTrails_1.json b/tests/data/placebo/test_s3_data_events/cloudtrail.DescribeTrails_1.json
new file mode 100644
--- /dev/null
+++ b/tests/data/placebo/test_s3_data_events/cloudtrail.DescribeTrails_1.json
@@ -0,0 +1,49 @@
+{
+    "status_code": 200,
+    "data": {
+        "trailList": [
+            {
+                "Name": "aws-controltower-BaselineCloudTrail",
+                "S3BucketName": "aws-controltower-logs-644160558196-us-east-1",
+                "S3KeyPrefix": "o-1234567890",
+                "SnsTopicName": "arn:aws:sns:us-east-1:644160558196:aws-controltower-AllConfigNotifications",
+                "SnsTopicARN": "arn:aws:sns:us-east-1:644160558196:aws-controltower-AllConfigNotifications",
+                "IncludeGlobalServiceEvents": true,
+                "IsMultiRegionTrail": true,
+                "HomeRegion": "us-east-1",
+                "TrailARN": "arn:aws:cloudtrail:us-east-1:644160558196:trail/aws-controltower-BaselineCloudTrail",
+                "LogFileValidationEnabled": true,
+                "CloudWatchLogsLogGroupArn": "arn:aws:logs:us-east-1:644160558196:log-group:aws-controltower/CloudTrailLogs:*",
+                "CloudWatchLogsRoleArn": "arn:aws:iam::644160558196:role/aws-controltower-CloudWatchLogsRole",
+                "HasCustomEventSelectors": false,
+                "HasInsightSelectors": false,
+                "IsOrganizationTrail": false
+            },
+            {
+                "Name": "data-events",
+                "S3BucketName": "aws-cloudtrail-logs-644160558196-ead919ec",
+                "IncludeGlobalServiceEvents": true,
+                "IsMultiRegionTrail": true,
+                "HomeRegion": "us-east-1",
+                "TrailARN": "arn:aws:cloudtrail:us-east-1:644160558196:trail/data-events",
+                "LogFileValidationEnabled": true,
+                "HasCustomEventSelectors": true,
+                "HasInsightSelectors": false,
+                "IsOrganizationTrail": false
+            },
+            {
+                "Name": "org-trail",
+                "S3BucketName": "aws-cloudtrail-logs-123456789012-3f556314",
+                "IncludeGlobalServiceEvents": true,
+                "IsMultiRegionTrail": true,
+                "HomeRegion": "us-east-1",
+                "TrailARN": "arn:aws:cloudtrail:us-east-1:123456789012:trail/org-trail",
+                "LogFileValidationEnabled": true,
+                "HasCustomEventSelectors": true,
+                "HasInsightSelectors": false,
+                "IsOrganizationTrail": true
+            }
+        ],
+        "ResponseMetadata": {}
+    }
+}
diff --git a/tests/data/placebo/test_s3_data_events/cloudtrail.GetEventSelectors_1.json b/tests/data/placebo/test_s3_data_events/cloudtrail.GetEventSelectors_1.json
new file mode 100644
--- /dev/null
+++ b/tests/data/placebo/test_s3_data_events/cloudtrail.GetEventSelectors_1.json
@@ -0,0 +1,15 @@
+{
+    "status_code": 200,
+    "data": {
+        "TrailARN": "arn:aws:cloudtrail:us-east-1:644160558196:trail/aws-controltower-BaselineCloudTrail",
+        "EventSelectors": [
+            {
+                "ReadWriteType": "All",
+                "IncludeManagementEvents": true,
+                "DataResources": [],
+                "ExcludeManagementEventSources": []
+            }
+        ],
+        "ResponseMetadata": {}
+    }
+}
\ No newline at end of file
diff --git a/tests/data/placebo/test_s3_data_events/cloudtrail.GetEventSelectors_2.json b/tests/data/placebo/test_s3_data_events/cloudtrail.GetEventSelectors_2.json
new file mode 100644
--- /dev/null
+++ b/tests/data/placebo/test_s3_data_events/cloudtrail.GetEventSelectors_2.json
@@ -0,0 +1,22 @@
+{
+    "status_code": 200,
+    "data": {
+        "TrailARN": "arn:aws:cloudtrail:us-east-1:644160558196:trail/data-events",
+        "EventSelectors": [
+            {
+                "ReadWriteType": "WriteOnly",
+                "IncludeManagementEvents": false,
+                "DataResources": [
+                    {
+                        "Type": "AWS::S3::Object",
+                        "Values": [
+                            "arn:aws:s3:::bucket-with-data-events/"
+                        ]
+                    }
+                ],
+                "ExcludeManagementEventSources": []
+            }
+        ],
+        "ResponseMetadata": {}
+    }
+}
diff --git a/tests/data/placebo/test_s3_data_events/s3.ListBuckets_1.json b/tests/data/placebo/test_s3_data_events/s3.ListBuckets_1.json
new file mode 100644
--- /dev/null
+++ b/tests/data/placebo/test_s3_data_events/s3.ListBuckets_1.json
@@ -0,0 +1,38 @@
+{
+    "status_code": 200,
+    "data": {
+        "ResponseMetadata": {},
+        "Buckets": [
+            {
+                "Name": "bucket-with-data-events",
+                "CreationDate": {
+                    "__class__": "datetime",
+                    "year": 2021,
+                    "month": 10,
+                    "day": 19,
+                    "hour": 18,
+                    "minute": 6,
+                    "second": 8,
+                    "microsecond": 0
+                }
+            },
+            {
+                "Name": "bucket-without-data-events",
+                "CreationDate": {
+                    "__class__": "datetime",
+                    "year": 2021,
+                    "month": 10,
+                    "day": 19,
+                    "hour": 15,
+                    "minute": 22,
+                    "second": 50,
+                    "microsecond": 0
+                }
+            }
+        ],
+        "Owner": {
+            "DisplayName": "aws-backoffice+sandbox-aj",
+            "ID": "e46bca81657fdc9f58bc5bab457564bb66f16636bd724a9f51d212fa2185a2b0"
+        }
+    }
+}
diff --git a/tests/data/placebo/test_s3_data_events/tagging.GetResources_1.json b/tests/data/placebo/test_s3_data_events/tagging.GetResources_1.json
new file mode 100644
--- /dev/null
+++ b/tests/data/placebo/test_s3_data_events/tagging.GetResources_1.json
@@ -0,0 +1,13 @@
+{
+    "status_code": 200,
+    "data": {
+        "PaginationToken": "",
+        "ResourceTagMappingList": [
+            {
+                "ResourceARN": "arn:aws:cloudtrail:us-east-1:644160558196:trail/aws-controltower-BaselineCloudTrail",
+                "Tags": []
+            }
+        ],
+        "ResponseMetadata": {}
+    }
+}
diff --git a/tests/test_s3.py b/tests/test_s3.py
--- a/tests/test_s3.py
+++ b/tests/test_s3.py
@@ -3739,6 +3739,23 @@ def test_s3_invoke_lambda_assume_role_action(self, mock_assumed_session):
 
         assert mock_assumed_session.call_count == 1
 
+    def test_s3_data_events(self):
+        self.patch(s3.S3, "executor_factory", MainThreadExecutor)
+        self.patch(s3, "S3_AUGMENT_TABLE", [])
+        session_factory = self.replay_flight_data("test_s3_data_events")
+
+        p = self.load_policy(
+            {
+                "name": "s3-data-events",
+                "resource": "s3",
+                "filters": [{"type": "data-events"}],
+            },
+            session_factory=session_factory,
+        )
+
+        resources = p.run()
+        assert {bucket["Name"] for bucket in resources} == {"bucket-with-data-events"}
+
 
 class S3LifecycleTest(BaseTest):