Repository: localstack/localstack

Issue:
bug: EventBridge event-pattern not correctly matched on detail content
### Is there an existing issue for this?

- [X] I have searched the existing issues

### Current Behavior

The provided example does not put a message in SQS.

### Expected Behavior

The provided example puts a message in SQS.

### How are you starting LocalStack?

With a docker-compose file

### Steps To Reproduce

The following scripts returns no items in SQS, but it should. The bit detail: foo is not matched.

```
awslocal events create-event-bus \
    --name detail-bus-1

awslocal sqs create-queue \
    --queue-name "detail-sqs-1"

awslocal events put-rule \
    --name "detail-rule-1" \
    --event-bus-name "detail-bus-1" \
    --event-pattern '{ "detail": { "foo": [ { "exists": true } ] } }'

awslocal events put-targets \
    --rule "detail-rule-1" \
    --event-bus-name "detail-bus-1" \
    --targets "Id"="detail-target-rule-1","Arn"="arn:aws:sqs:us-east-1:000000000000:detail-sqs-1","InputPath"="$.detail"

awslocal events put-events --entries '[{"DetailType": "test", "Source": "aws.test", "Detail": "{ \"foo\": \"bar\" }", "EventBusName": "detail-bus-1"}]'
awslocal sqs receive-message --queue-url http://localhost:4566/000000000000/detail-sqs-1
```

### Environment

```markdown
- OS: macOS 13.3.1
- LocalStack: 1.4.0
```


### Anything else?

_No response_
bug: Bug with EventBus filters
### Is there an existing issue for this?

- [X] I have searched the existing issues

### Current Behavior

I wanted to test an EventBus filter that will trigger a Lambda only when a field doesn't exist or when exists and equals to false.
So I created a Lambda with the following configuration (using Serverless Framework)
This is my Lambda Configuration
```  
test:
    handler: test.handler
    memorySize: 512
    description: test
    events:
      - eventBridge:
          eventBus: ${ssm:/${self:custom.env_name}/infra/event-bus/test, 'default'}
          pattern:
            source:
              - 'other-service'
            detail-type:
              - 'test'
            detail:
              event_name:
                - 'test_event_name'
              test_field:
                - false
       - eventBridge:
            eventBus: ${ssm:/${self:custom.env_name}/infra/event-bus/test, 'default'}
            pattern:
              source:
                - 'other-service'
              detail-type:
                - 'test'
              detail:
                event_name:
                  - 'test_event_name'
                test_field:
                    is_background_job: [{exists: false}]
```

When I push an event, It doesn't trigger the Lambda. 
When I tested it on AWS, it worked perfectly.



### Expected Behavior

To trigger the Lambda

### How are you starting LocalStack?

With a docker-compose file

### Steps To Reproduce

  localstack:
    image: localstack/localstack:1.3.0
    ports:
      - "4566-4583:4566-4583"
      - "${PORT_WEB_UI-4666}:${PORT_WEB_UI-8080}"
      - "8080:8080"
      - "4510:4510"
    environment:
      - LOCALSTACK_API_KEY=${LOCALSTACK_API_KEY- }
      - PORT_WEB_UI=8080
      - PERSISTENCE=0
      - LAMBDA_REMOTE_DOCKER=0
      - LAMBDA_EXECUTOR=docker-reuse
      - LAMBDA_REMOVE_CONTAINERS=true
      - DOCKER_HOST=unix:///var/run/docker.sock
      - LAMBDA_DOCKER_NETWORK=localstack-net
      - KINESIS_ERROR_PROBABILITY=${KINESIS_ERROR_PROBABILITY- }
      - LS_LOG=debug
      - DEBUG=1
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock"
      - "${LOCALSTACK_VOLUME_DIR:-~/Library/Caches/localstack/volume}:/var/lib/localstack"
    networks:
      - localstack-net

### Environment

```markdown
- OS: ubuntu 20.04
- LocalStack: 1.3.0
```


### Anything else?
I tried to deploy it without the test_field filter:
```
test:
    handler: test.handler
    memorySize: 512
    description: test
    events:
      - eventBridge:
          eventBus: ${ssm:/${self:custom.env_name}/infra/event-bus/test, 'default'}
          pattern:
            source:
              - 'other-service'
            detail-type:
              - 'test'
            detail:
              event_name:
                - 'test_event_name'
 ```
And it worked well. 



Discussion:
Welcome to LocalStack! Thanks for reporting your first issue and our team will be working towards fixing the issue for you or reach out for more background information. We recommend joining our [Slack Community](https://localstack.cloud/contact/) for real-time help and drop a message to LocalStack Pro Support if you are a Pro user! If you are willing to contribute towards fixing this issue, please have a look at our [contributing guidelines](https://github.com/localstack/.github/blob/main/CONTRIBUTING.md) and our [contributing guide](https://docs.localstack.cloud/contributing/).
I've noticed this too, the below rule matches when the property is an exact match, and the rule is explicit:
```json
{"detail":{"payload":{"callbackUrl":["https://actual-url-to-match"]}}}
```

But simply checking if the property exists no longer matches:
```json
{"detail":{"payload":{"callbackUrl":[{"exists":true}]}}}
```

This rule works fine on AWS itself, but doesn't appear to work in Localstack.
Tested on Docker:
```
LocalStack version: 3.0.0
LocalStack Docker container id: e60c81af69e7
LocalStack build date: 2023-11-16
LocalStack build git hash: 3cd32364
```

The relevant AWS Documentation for the exists-matching is here:
https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-event-patterns-content-based-filtering.html#eb-filtering-exists-matching
Looks like this issue may be the same as #7305.
We're also bumping into this bug

## Environment
- OS: macOS 13.0 (Ventura)
- LocalStack
```
2023-04-28 14:11:07 LocalStack version: 2.0.3.dev20230424170116
2023-04-28 14:11:07 LocalStack build date: 2023-04-26
2023-04-28 14:11:07 LocalStack build git hash: 48fa428
```
## Steps to repro
- Set up an event bus
- Set up a lambda function
- Add a rule to the event bus with a pattern that includes an `exists` condition with the lambda function as the target
- Send an event to the event bus that matched the pattern

### Expected behavior
Lambda function is invoked

Verified that the event pattern and event produced a match via the [AWS EventBridge console](https://us-east-1.console.aws.amazon.com/events/home?region=us-east-1#/explore)

### Actual behavior
Lambda function is not invoked

If the `exists` condition is removed from the event pattern, the lambda function is invoked correctly.

## Sample data

### Event pattern
```json
{
  "source": ["source"],
  "detail": {
    "payload": {
      "key": [{
        "exists": false
      }]
    }
  }
}
```

### Event
```json
{
  "version": "0",
  "id": "f1aa4e97-5226-4ed7-b116-4e4b996ab6d1",
  "detail-type": "EVENT",
  "source": "source",
  "account": "000000000000",
  "time": "2023-04-28T19:48:40Z",
  "region": "us-east-1",
  "resources": [],
  "detail": {
    "payload": {
      "anotherKey": 12345
    }
  }
}
```

![Screenshot 2023-04-28 at 2 49 35 PM](https://user-images.githubusercontent.com/27798268/235260054-71ee6786-2148-494f-99d2-1cdec2907ef9.png)


I'm also running into this issue with "exists"

Resolving patch:
diff --git a/localstack/services/events/provider.py b/localstack/services/events/provider.py
--- a/localstack/services/events/provider.py
+++ b/localstack/services/events/provider.py
@@ -376,7 +376,7 @@ def _dump_events_to_files(events_with_added_uuid):
         LOG.info("Unable to dump events to tmp dir %s: %s", _get_events_tmp_dir(), e)
 
 
-def handle_numeric_conditions(conditions: List[Any], value: float):
+def handle_numeric_conditions(conditions: list[Any], value: float):
     for i in range(0, len(conditions), 2):
         if conditions[i] == "<" and not (value < conditions[i + 1]):
             return False
@@ -423,7 +423,7 @@ def filter_event_with_content_base_parameter(pattern_value: list, event_value: s
             elif element_key.lower() == "exists":
                 if element_value and event_value:
                     return True
-                elif not element_value and not event_value:
+                elif not element_value and isinstance(event_value, object):
                     return True
             elif element_key.lower() == "cidr":
                 ips = [str(ip) for ip in ipaddress.IPv4Network(element_value)]
@@ -489,6 +489,9 @@ def handle_prefix_filtering(event_pattern, value):
         elif isinstance(element, dict) and "anything-but" in element:
             if element.get("anything-but") != value:
                 return True
+        elif isinstance(element, dict) and "exists" in element:
+            if element.get("exists") and value:
+                return True
         elif "numeric" in element:
             return handle_numeric_conditions(element.get("numeric"), value)
         elif isinstance(element, list):
@@ -510,15 +513,23 @@ def get_two_lists_intersection(lst1: List, lst2: List) -> List:
     return lst3
 
 
+def event_pattern_prefix_bool_filter(event_pattern_filter_value_list: list[dict[str, Any]]) -> bool:
+    for event_pattern_filter_value in event_pattern_filter_value_list:
+        if "exists" in event_pattern_filter_value:
+            return event_pattern_filter_value.get("exists")
+        else:
+            return True
+
+
 # TODO: refactor/simplify!
 def filter_event_based_on_event_format(
-    self, rule_name: str, event_bus_name: str, event: Dict[str, Any]
+    self, rule_name: str, event_bus_name: str, event: dict[str, Any]
 ):
-    def filter_event(event_pattern_filter: Dict[str, Any], event: Dict[str, Any]):
+    def filter_event(event_pattern_filter: dict[str, Any], event: dict[str, Any]):
         for key, value in event_pattern_filter.items():
             fallback = object()
             event_value = event.get(key.lower(), event.get(key, fallback))
-            if event_value is fallback:
+            if event_value is fallback and event_pattern_prefix_bool_filter(value):
                 return False
 
             # 1. check if certain values in the event do not match the expected pattern
@@ -584,7 +595,7 @@ def filter_event_with_target_input_path(target: Dict, event: Dict) -> Dict:
     return event
 
 
-def process_events(event: Dict, targets: List[Dict]):
+def process_events(event: Dict, targets: list[Dict]):
     for target in targets:
         arn = target["Arn"]
         changed_event = filter_event_with_target_input_path(target, event)


Test patch:
diff --git a/tests/aws/services/events/test_events.py b/tests/aws/services/events/test_events.py
--- a/tests/aws/services/events/test_events.py
+++ b/tests/aws/services/events/test_events.py
@@ -397,7 +397,7 @@ def _put_events_with_filter_to_sqs(
                 for entry_asserts in entries_asserts:
                     entries = entry_asserts[0]
                     for entry in entries:
-                        entry.setdefault("EventBusName", bus_name)
+                        entry["EventBusName"] = bus_name
                     message = self._put_entries_assert_results_sqs(
                         events_client,
                         sqs_client,
@@ -426,13 +426,19 @@ def _put_entries_assert_results_sqs(
         assert not response.get("FailedEntryCount")
 
         def get_message(queue_url):
-            resp = sqs_client.receive_message(QueueUrl=queue_url)
+            resp = sqs_client.receive_message(
+                QueueUrl=queue_url, WaitTimeSeconds=5, MaxNumberOfMessages=1
+            )
             messages = resp.get("Messages")
+            if messages:
+                for message in messages:
+                    receipt_handle = message["ReceiptHandle"]
+                    sqs_client.delete_message(QueueUrl=queue_url, ReceiptHandle=receipt_handle)
             if should_match:
                 assert len(messages) == 1
             return messages
 
-        messages = retry(get_message, retries=5, sleep=1, queue_url=queue_url)
+        messages = retry(get_message, retries=5, queue_url=queue_url)
 
         if should_match:
             actual_event = json.loads(messages[0]["Body"])
@@ -497,6 +503,100 @@ def test_put_events_with_rule_anything_but_to_sqs(
         )
         snapshot.match("rule-anything-but", messages)
 
+    @markers.aws.validated
+    def test_put_events_with_rule_exists_true_to_sqs(self, put_events_with_filter_to_sqs, snapshot):
+        """
+        Exists matching True condition: https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-event-patterns-content-based-filtering.html#eb-filtering-exists-matching
+        """
+        snapshot.add_transformer(
+            [
+                snapshot.transform.key_value("MD5OfBody"),
+                snapshot.transform.key_value("ReceiptHandle"),
+                snapshot.transform.jsonpath("$..EventBusName", "event-bus-name"),
+            ]
+        )
+
+        event_detail_exists = {"key": "value", "payload": "baz"}
+        event_detail_not_exists = {"no-key": "no-value", "payload": "baz"}
+        event_patter_details = ["core.update-account-command"]
+        test_event_pattern_exists = {
+            "source": event_patter_details,
+            "detail-type": event_patter_details,
+            "detail": {"key": [{"exists": True}]},
+        }
+        entries_exists = [
+            {
+                "Source": test_event_pattern_exists["source"][0],
+                "DetailType": test_event_pattern_exists["detail-type"][0],
+                "Detail": json.dumps(event_detail_exists),
+            }
+        ]
+        entries_not_exists = [
+            {
+                "Source": test_event_pattern_exists["source"][0],
+                "DetailType": test_event_pattern_exists["detail-type"][0],
+                "Detail": json.dumps(event_detail_not_exists),
+            }
+        ]
+        entries_asserts = [
+            (entries_exists, True),
+            (entries_not_exists, False),
+        ]
+
+        messages = put_events_with_filter_to_sqs(
+            pattern=test_event_pattern_exists,
+            entries_asserts=entries_asserts,
+        )
+        snapshot.match("rule-exists-true", messages)
+
+    @markers.aws.validated
+    def test_put_events_with_rule_exists_false_to_sqs(
+        self, put_events_with_filter_to_sqs, snapshot
+    ):
+        """
+        Exists matching False condition: https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-event-patterns-content-based-filtering.html#eb-filtering-exists-matching
+        """
+        snapshot.add_transformer(
+            [
+                snapshot.transform.key_value("MD5OfBody"),
+                snapshot.transform.key_value("ReceiptHandle"),
+                snapshot.transform.jsonpath("$..EventBusName", "event-bus-name"),
+            ]
+        )
+
+        event_detail_exists = {"key": "value", "payload": "baz"}
+        event_detail_not_exists = {"no-key": "no-value", "payload": "baz"}
+        event_patter_details = ["core.update-account-command"]
+        test_event_pattern_not_exists = {
+            "source": event_patter_details,
+            "detail-type": event_patter_details,
+            "detail": {"key": [{"exists": False}]},
+        }
+        entries_exists = [
+            {
+                "Source": test_event_pattern_not_exists["source"][0],
+                "DetailType": test_event_pattern_not_exists["detail-type"][0],
+                "Detail": json.dumps(event_detail_exists),
+            }
+        ]
+        entries_not_exists = [
+            {
+                "Source": test_event_pattern_not_exists["source"][0],
+                "DetailType": test_event_pattern_not_exists["detail-type"][0],
+                "Detail": json.dumps(event_detail_not_exists),
+            }
+        ]
+        entries_asserts_exists_false = [
+            (entries_exists, False),
+            (entries_not_exists, True),
+        ]
+
+        messages_not_exists = put_events_with_filter_to_sqs(
+            pattern=test_event_pattern_not_exists,
+            entries_asserts=entries_asserts_exists_false,
+        )
+        snapshot.match("rule-exists-false", messages_not_exists)
+
     # TODO: further unify/parameterize the tests for the different target types below
 
     @markers.aws.unknown
diff --git a/tests/aws/services/events/test_events.snapshot.json b/tests/aws/services/events/test_events.snapshot.json
--- a/tests/aws/services/events/test_events.snapshot.json
+++ b/tests/aws/services/events/test_events.snapshot.json
@@ -292,5 +292,57 @@
         }
       ]
     }
+  },
+  "tests/aws/services/events/test_events.py::TestEvents::test_put_events_with_rule_exists_true_to_sqs": {
+    "recorded-date": "29-12-2023, 09:56:47",
+    "recorded-content": {
+      "rule-exists-true": [
+        {
+          "MessageId": "<uuid:1>",
+          "ReceiptHandle": "<receipt-handle:1>",
+          "MD5OfBody": "<m-d5-of-body:1>",
+          "Body": {
+            "version": "0",
+            "id": "<uuid:2>",
+            "detail-type": "core.update-account-command",
+            "source": "core.update-account-command",
+            "account": "111111111111",
+            "time": "date",
+            "region": "<region>",
+            "resources": [],
+            "detail": {
+              "key": "value",
+              "payload": "baz"
+            }
+          }
+        }
+      ]
+    }
+  },
+  "tests/aws/services/events/test_events.py::TestEvents::test_put_events_with_rule_exists_false_to_sqs": {
+    "recorded-date": "29-12-2023, 09:59:20",
+    "recorded-content": {
+      "rule-exists-false": [
+        {
+          "MessageId": "<uuid:1>",
+          "ReceiptHandle": "<receipt-handle:1>",
+          "MD5OfBody": "<m-d5-of-body:1>",
+          "Body": {
+            "version": "0",
+            "id": "<uuid:2>",
+            "detail-type": "core.update-account-command",
+            "source": "core.update-account-command",
+            "account": "111111111111",
+            "time": "date",
+            "region": "<region>",
+            "resources": [],
+            "detail": {
+              "no-key": "no-value",
+              "payload": "baz"
+            }
+          }
+        }
+      ]
+    }
   }
 }
diff --git a/tests/aws/services/events/test_events.validation.json b/tests/aws/services/events/test_events.validation.json
--- a/tests/aws/services/events/test_events.validation.json
+++ b/tests/aws/services/events/test_events.validation.json
@@ -8,6 +8,12 @@
   "tests/aws/services/events/test_events.py::TestEvents::test_put_events_to_default_eventbus_for_custom_eventbus": {
     "last_validated_date": "2022-12-22T00:50:01+00:00"
   },
+  "tests/aws/services/events/test_events.py::TestEvents::test_put_events_with_rule_exists_false_to_sqs": {
+    "last_validated_date": "2023-12-29T09:59:20+00:00"
+  },
+  "tests/aws/services/events/test_events.py::TestEvents::test_put_events_with_rule_exists_true_to_sqs": {
+    "last_validated_date": "2023-12-29T09:56:47+00:00"
+  },
   "tests/aws/services/events/test_events.py::TestEvents::test_put_rule": {
     "last_validated_date": "2023-09-29T17:26:53+00:00"
   },