Repository: deepset-ai/haystack

Issue:
Meta data filtering with InMemory doc store and Embedding Retriever
**Describe the bug**
Meta-data filtering during retrieval is returning no documents when used with the Embedding Retriever, InMemory doc store, and the Entity Extractor. 
 
**Error message**
No error message. Just no returned document(s).

**Expected behavior**
At least one document should be returned for each query using the filters to search only on documents that share similar meta-data values for the "entity_words" field.

**Additional context**
With the Embedding Retriever and the InMemory document store, I want to use the Entity Extractor to 1) add NER as meta-data to my indexed documents 2) Perform NER from my query to obtain a list of entities 3) Use this list of entities extracted from my query + the "entity_words" meta-data to filter on meta-data during the retrieval process. 

I have confirmed that the retrieval pipeline works correctly without filters. I have also tried various filter values with no luck.

**To Reproduce**
Set up entity extractor:

```
entity_extractor = EntityExtractor(model_name_or_path="elastic/distilbert-base-cased-finetuned-conll03-english", use_gpu=True, flatten_entities_in_meta_data=True)

```
Run the indexing pipeline as instructed in the documentation [https://docs.haystack.deepset.ai/docs/entity_extractor]

```
pipeline = Pipeline()
pipeline.add_node(text_converter, "TextConverter", ["File"])
pipeline.add_node(preprocessor, "PreProcessor", ["TextConverter"])
pipeline.add_node(entity_extractor, "EntityExtractor", ["PreProcessor"])
pipeline.add_node(document_store, "DocumentStore", ["EntityExtractor"])
pipeline.run(file_paths=file_paths) 

```

Index documents into the InMemory document store. At this point, the documents look correctly indexed when inspected, including the expected 5 meta-data fields from the Entity Extractor: entity_groups, entity_scores, entity_words, entity_starts, and entity_ends.

During retrieval, perform entity extraction on the query to extract the entities relevant to the query. Output of this step is a list of entities. Here is the code I use for that: 

```
ent_r = (entity_extractor.extract(query))
ent = flatten(ent_r)

```
where flatten() is borrowed from the source code as a custom function to shape the output from the entity extractor as a list that's equivalent to the "entity_words":

```
def flatten(entities):
    new_key_map = {
            "entity_group": "entity_groups",
            "score": "entity_scores",
            "word": "entity_words",
            "start": "entity_starts",
            "end": "entity_ends",
        }
    entity_lists: Dict[str, List[Any]] = {v: [] for k, v in new_key_map.items()}
    for entity in entities:
        for key in entity:
            new_key = new_key_map[key]
            if isinstance(entity[key], np.float32):
                entity_lists[new_key].append(float(entity[key]))
            else:
                entity_lists[new_key].append(entity[key])
    return entity_lists
```

The above part works great as well. The output is, as expected, a list of entities based on the query. 

During the actual retrieval, add a filter that should use the list of extracted entities from the query to filter against the "entity_words" field of the indexed documents:

```
   result = pipeline.run(
        query = query,
        params={
            "top_k": top_k,
            "filters": {
                "entity_words": ent['entity_words'],
            }
        },
    )
```

This is the part that fails, returning no documents. If the filter is removed, the pipeline works correctly.

**FAQ Check**
- [X ] Have you had a look at [our new FAQ page](https://docs.haystack.deepset.ai/docs/faq)? Yep!

**System:**
 - OS: Linux
 - GPU/CPU: GPU
 - Haystack version (commit or version number): 1.17.1
 - DocumentStore: InMemory
 - Reader: N/A
 - Retriever + Ranker: Embedding Retriever + Ranker



Discussion:

Resolving patch:
diff --git a/haystack/document_stores/filter_utils.py b/haystack/document_stores/filter_utils.py
--- a/haystack/document_stores/filter_utils.py
+++ b/haystack/document_stores/filter_utils.py
@@ -482,8 +482,15 @@ class InOperation(ComparisonOperation):
     def evaluate(self, fields) -> bool:
         if self.field_name not in fields:
             return False
-        return fields[self.field_name] in self.comparison_value  # type: ignore
-        # is only initialized with lists, but changing the type annotation would mean duplicating __init__
+
+        if not isinstance(self.comparison_value, list):
+            raise FilterError("'$in' operation requires comparison value to be a list.")
+
+        # If the document field is a list, check if any of its values are in the comparison value
+        if isinstance(fields[self.field_name], list):
+            return any(field in self.comparison_value for field in fields[self.field_name])
+
+        return fields[self.field_name] in self.comparison_value
 
     def convert_to_elasticsearch(self) -> Dict[str, Dict[str, List]]:
         if not isinstance(self.comparison_value, list):
@@ -563,9 +570,16 @@ class NinOperation(ComparisonOperation):
 
     def evaluate(self, fields) -> bool:
         if self.field_name not in fields:
-            return False
-        return fields[self.field_name] not in self.comparison_value  # type: ignore
-        # is only initialized with lists, but changing the type annotation would mean duplicating __init__
+            return True
+
+        if not isinstance(self.comparison_value, list):
+            raise FilterError("'$nin' operation requires comparison value to be a list.")
+
+        # If the document field is a list, check if any of its values are in the comparison value
+        if isinstance(fields[self.field_name], list):
+            return not any(field in self.comparison_value for field in fields[self.field_name])
+
+        return fields[self.field_name] not in self.comparison_value
 
     def convert_to_elasticsearch(self) -> Dict[str, Dict[str, Dict[str, Dict[str, List]]]]:
         if not isinstance(self.comparison_value, list):
@@ -611,6 +625,11 @@ class GtOperation(ComparisonOperation):
     def evaluate(self, fields) -> bool:
         if self.field_name not in fields:
             return False
+
+        # If the document field is a list, check if any of its values are greater than the comparison value
+        if isinstance(fields[self.field_name], list):
+            return any(field > self.comparison_value for field in fields[self.field_name])
+
         return fields[self.field_name] > self.comparison_value
 
     def convert_to_elasticsearch(self) -> Dict[str, Dict[str, Dict[str, Union[str, float, int]]]]:
@@ -650,6 +669,11 @@ class GteOperation(ComparisonOperation):
     def evaluate(self, fields) -> bool:
         if self.field_name not in fields:
             return False
+
+        # If the document field is a list, check if any of its values are greater than or equal to the comparison value
+        if isinstance(fields[self.field_name], list):
+            return any(field >= self.comparison_value for field in fields[self.field_name])
+
         return fields[self.field_name] >= self.comparison_value
 
     def convert_to_elasticsearch(self) -> Dict[str, Dict[str, Dict[str, Union[str, float, int]]]]:
@@ -689,6 +713,11 @@ class LtOperation(ComparisonOperation):
     def evaluate(self, fields) -> bool:
         if self.field_name not in fields:
             return False
+
+        # If the document field is a list, check if any of its values are less than the comparison value
+        if isinstance(fields[self.field_name], list):
+            return any(field < self.comparison_value for field in fields[self.field_name])
+
         return fields[self.field_name] < self.comparison_value
 
     def convert_to_elasticsearch(self) -> Dict[str, Dict[str, Dict[str, Union[str, float, int]]]]:
@@ -728,6 +757,11 @@ class LteOperation(ComparisonOperation):
     def evaluate(self, fields) -> bool:
         if self.field_name not in fields:
             return False
+
+        # If the document field is a list, check if any of its values are less than or equal to the comparison value
+        if isinstance(fields[self.field_name], list):
+            return any(field <= self.comparison_value for field in fields[self.field_name])
+
         return fields[self.field_name] <= self.comparison_value
 
     def convert_to_elasticsearch(self) -> Dict[str, Dict[str, Dict[str, Union[str, float, int]]]]:


Test patch:
diff --git a/test/document_stores/test_memory.py b/test/document_stores/test_memory.py
--- a/test/document_stores/test_memory.py
+++ b/test/document_stores/test_memory.py
@@ -36,26 +36,6 @@ def test_ne_filters(self, ds, documents):
         result = ds.get_all_documents(filters={"year": {"$ne": "2020"}})
         assert len(result) == 3
 
-    @pytest.mark.skip
-    @pytest.mark.integration
-    def test_nin_filters(self, ds, documents):
-        pass
-
-    @pytest.mark.skip
-    @pytest.mark.integration
-    def test_comparison_filters(self, ds, documents):
-        pass
-
-    @pytest.mark.skip
-    @pytest.mark.integration
-    def test_nested_condition_filters(self, ds, documents):
-        pass
-
-    @pytest.mark.skip
-    @pytest.mark.integration
-    def test_nested_condition_not_filters(self, ds, documents):
-        pass
-
     @pytest.mark.integration
     def test_get_documents_by_id(self, ds, documents):
         """