Repository: cloud-custodian/cloud-custodian

Issue:
aws - wafv2 raising TypeError when waf has ByteMatchStatement Rules
### Describe the bug

I have some AWS Web ACLs that have custom rules that use ByteMatchStatement, and when I use the following policy it returns the error defined in "Relevant log/traceback output" section. From what I investigated and show below, AWS response returns some information in "bytes" type that leads to an Error during json.dumps.

I added a line in the function utils.dumps to have the full picture of the python dict before the JSON dumps.
```python
def dumps(data, fh=None, indent=0):
    log.debug(data)
    if fh:
        return json.dump(data, fh, cls=DateTimeEncoder, indent=indent)
    else:
        return json.dumps(data, cls=DateTimeEncoder, indent=indent)
``` 

WAFv2 result object before json.dumps - it's possible to see that Rules[].Statement.ByteMatchStatement.SearchString is in "bytes" type.
```python
[
    {
        'Scope': 'REGIONAL',
        'Name': 'WAF_NAME',
        'Id': 'WAF_ID',
        'Description': 'WAFv2 ACL',
        'LockToken': 'LOCK_TOKEN',
        'ARN': 'WEB_ACL_ARN',
        'Tags': [],
        'DefaultAction': {'Block': {}},
        'Rules': [
            {
                'Name': 'rule-name',
                'Priority': 1,
                'Statement': {
                    'ByteMatchStatement': {
                        'SearchString': b'/path/to/url',
                        'FieldToMatch': {'UriPath': {}},
                        'TextTransformations': [{'Priority': 0, 'Type': 'NONE'}],
                        'PositionalConstraint': 'STARTS_WITH',
                    }
                },
                'Action': {'Allow': {}},
                'VisibilityConfig': {
                    'SampledRequestsEnabled': True,
                    'CloudWatchMetricsEnabled': True,
                    'MetricName': 'metric-name',
                },
            }
      ]
  }
]
```

### What did you expect to happen?

It's expected that Rules[].Statement.ByteMatchStatement.SearchString to be **str** and not **bytes** otherwise json.dump will fail.

### Cloud Provider

Amazon Web Services (AWS)

### Cloud Custodian version and dependency information

```shell
0.9.30
```


### Policy

```shell
policies:
  - name: notify-waf-without-common-ruleset
    description: |
      Notify when waf does not have the common rule set
    resource: wafv2
    mode:
      schedule: "cron(0 9 ? * TUE *)"
      type: periodic
      role: CUSTODIAN_ROLE
      execution-options:
        output_dir: s3://mybucket/CustodianLogs/
```


### Relevant log/traceback output

```shell
2023-08-21 15:21:05,949: custodian.commands:ERROR Error while executing policy notify-waf-without-common-ruleset, continuing
Traceback (most recent call last):
  File "/home/rubsilva/cloud-custodian/c7n/commands.py", line 307, in run
    policy()
  File "/home/rubsilva/cloud-custodian/c7n/policy.py", line 1352, in __call__
    resources = PullMode(self).run()
  File "/home/rubsilva/cloud-custodian/c7n/policy.py", line 351, in run
    ctx.output.write_file('resources.json', utils.dumps(resources, indent=2))
  File "/home/rubsilva/cloud-custodian/c7n/utils.py", line 101, in dumps
    return json.dumps(data, cls=DateTimeEncoder, indent=indent)
  File "/usr/lib/python3.8/json/__init__.py", line 234, in dumps
    return cls(
  File "/usr/lib/python3.8/json/encoder.py", line 201, in encode
    chunks = list(chunks)
  File "/usr/lib/python3.8/json/encoder.py", line 429, in _iterencode
    yield from _iterencode_list(o, _current_indent_level)
  File "/usr/lib/python3.8/json/encoder.py", line 325, in _iterencode_list
    yield from chunks
  File "/usr/lib/python3.8/json/encoder.py", line 405, in _iterencode_dict
    yield from chunks
  File "/usr/lib/python3.8/json/encoder.py", line 325, in _iterencode_list
    yield from chunks
  File "/usr/lib/python3.8/json/encoder.py", line 405, in _iterencode_dict
    yield from chunks
  File "/usr/lib/python3.8/json/encoder.py", line 405, in _iterencode_dict
    yield from chunks
  File "/usr/lib/python3.8/json/encoder.py", line 405, in _iterencode_dict
    yield from chunks
  File "/usr/lib/python3.8/json/encoder.py", line 438, in _iterencode
    o = _default(o)
  File "/home/rubsilva/cloud-custodian/c7n/utils.py", line 223, in default
    return json.JSONEncoder.default(self, obj)
  File "/usr/lib/python3.8/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type bytes is not JSON serializable
```


### Extra information or context

_No response_


Discussion:

Resolving patch:
diff --git a/c7n/utils.py b/c7n/utils.py
--- a/c7n/utils.py
+++ b/c7n/utils.py
@@ -95,9 +95,9 @@ def loads(body):
 
 def dumps(data, fh=None, indent=0):
     if fh:
-        return json.dump(data, fh, cls=DateTimeEncoder, indent=indent)
+        return json.dump(data, fh, cls=JsonEncoder, indent=indent)
     else:
-        return json.dumps(data, cls=DateTimeEncoder, indent=indent)
+        return json.dumps(data, cls=JsonEncoder, indent=indent)
 
 
 def format_event(evt):
@@ -212,13 +212,15 @@ def type_schema(
     return s
 
 
-class DateTimeEncoder(json.JSONEncoder):
+class JsonEncoder(json.JSONEncoder):
 
     def default(self, obj):
         if isinstance(obj, datetime):
             return obj.isoformat()
         if isinstance(obj, FormatDate):
             return obj.datetime.isoformat()
+        if isinstance(obj, bytes):
+            return obj.decode('utf8', errors="ignore")
         return json.JSONEncoder.default(self, obj)
 
 


Test patch:
diff --git a/tests/test_utils.py b/tests/test_utils.py
--- a/tests/test_utils.py
+++ b/tests/test_utils.py
@@ -215,6 +215,13 @@ def test_local_session_region(self):
 
         self.assertEqual(utils.local_session(p.session_factory), previous)
 
+    def test_encode_bytes(self):
+        self.assertEqual(
+            json.loads(json.dumps(
+                {"bytes": b"123"}, cls=utils.JsonEncoder)),
+                {'bytes': '123'}
+            )
+
     def test_format_date(self):
         d = parse_date("2018-02-02 12:00")
         self.assertEqual("{}".format(utils.FormatDate(d)), "2018-02-02 12:00:00")
@@ -228,7 +235,7 @@ def test_format_date(self):
         self.assertEqual("{:+5M%M}".format(utils.FormatDate(d)), "05")
 
         self.assertEqual(json.dumps(utils.FormatDate(d),
-                                    cls=utils.DateTimeEncoder, indent=2),
+                                    cls=utils.JsonEncoder, indent=2),
                          '"2018-02-02T12:00:00"')
         self.assertEqual(str(d), '2018-02-02 12:00:00')
 
@@ -438,7 +445,7 @@ def test_format_event(self):
         self.assertEqual(json.loads(utils.format_event(event)), json.loads(event_json))
 
     def test_date_time_decoder(self):
-        dtdec = utils.DateTimeEncoder()
+        dtdec = utils.JsonEncoder()
         self.assertRaises(TypeError, dtdec.default, "test")
 
     def test_set_annotation(self):