Repository: deepset-ai/haystack
Pull request: feat: Add options for what to do with missing metadata fields in `MetaFieldRanker`
### Related Issues

- fixes #7691 
- Adds param `missing_meta` to `MetaFieldRanker`:
```
        :param missing_meta:
            What to do with documents that are missing the sorting metadata field.
            Possible values are:
            - 'drop' will drop the documents entirely.
            - 'top' will place the documents at the top of the metadata-sorted list
                (regardless of 'ascending' or 'descending').
            - 'bottom' will place the documents at the bottom of metadata-sorted list
                (regardless of 'ascending' or 'descending').
```

### Proposed Changes:

- The `missing_meta` param has three options: `"bottom"`, `"top"`, and `"drop"`.
  - Using `"bottom"` exhibits the same behavior as was implemented prior to this PR, i.e., documents without the sorting metadata field are put on the bottom of the sorted list.
  - Using `"top"` puts them at the top instead.
  - Using `"drop"` drops such documents entirely.
- Validation was added to ensure that the value of `missing_meta` is legit.
- Tests were added for the new functionality.

### How did you test it?

Wrote and tried new tests functions in the `test` directory:
- `test_raises_value_error_if_wrong_missing_meta`: Tests validation of `missing_meta`
- `test_missing_meta_bottom`: Tests that `missing_meta = "bottom"` behaves as desired.
- `test_missing_meta_top`: Tests that `missing_meta = "top"` behaves as desired.
- `test_missing_meta_drop`: Tests that `missing_meta = "drop"` behaves as desired.

### Notes for the reviewer

<!-- E.g. point out section where the reviewer  -->
_None_

### Checklist

- [X] I have read the [contributors guidelines](https://github.com/deepset-ai/haystack/blob/main/CONTRIBUTING.md) and the [code of conduct](https://github.com/deepset-ai/haystack/blob/main/code_of_conduct.txt)
- [ ] I have updated the related issue with new insights and changes
- [X] I added unit tests and updated the docstrings
- [X] I've used one of the [conventional commit types](https://www.conventionalcommits.org/en/v1.0.0/) for my PR title: `fix:`, `feat:`, `build:`, `chore:`, `ci:`, `docs:`, `style:`, `refactor:`, `perf:`, `test:`.
- [X] I documented my code
- [X] I ran [pre-commit hooks](https://github.com/deepset-ai/haystack/blob/main/CONTRIBUTING.md#installation) and fixed any issue


Problem statement:
`MetaFieldRanker`: allow different options for what to do with missing metadata field
**Is your feature request related to a problem? Please describe.**
This is not an immediately faced problem, but seems like something that would be undesired in the long run. Long story short, currently when `MetaFieldRanker` sorts by the assigned metadata field, it puts any documents that are missing that metadata field at the bottom of that sort (assuming there is at least one doc that does have that metadata field). So e.g. if sorting by a date metadata field in descending order, docs without that metadata field are treated as chronologically prior to all docs with that metadata field. This may not be the desired way to handle this situation in all use cases.

**Describe the solution you'd like**
I think it'd make sense to have a parameter like `missing_meta: Literal["bottom", "top", "drop"] = "bottom"`, which determines what to do with documents that are missing the sort metadata field.
- `"bottom"` (default): documents with missing metadata field are sorted at **bottom** when sorting for that metadata field (currently-exhibited behavior).
- `"top"`: documents with missing metadata field are sorted at **top** when sorting for that metadata field.
- `"drop"`: documents with missing metadata field are dropped entirely.

I'm certainly open to alternatives, but these seemed to me to be the three most obvious options.

**Describe alternatives you've considered**
Haven't seriously considered other options. The next best thing as far as I can tell would be to introduce an additional component that does some filtering or re-ranking, so that way `MetaFieldRanker` stays intact but these alternative options are achievable. This seems to me to be a worse option, however.

**Additional context**
I'd love to take this on myself.

Hints from issue discussion:

Commit message: Merge branch 'main' into metf_optionality

Patch to review:
diff --git a/haystack/components/rankers/meta_field.py b/haystack/components/rankers/meta_field.py
index 964d4db974..5a91659de6 100644
--- a/haystack/components/rankers/meta_field.py
+++ b/haystack/components/rankers/meta_field.py
@@ -43,6 +43,7 @@ def __init__(
         top_k: Optional[int] = None,
         ranking_mode: Literal["reciprocal_rank_fusion", "linear_score"] = "reciprocal_rank_fusion",
         sort_order: Literal["ascending", "descending"] = "descending",
+        missing_meta: Literal["drop", "top", "bottom"] = "bottom",
         meta_value_type: Optional[Literal["float", "int", "date"]] = None,
     ):
         """
@@ -65,6 +66,14 @@ def __init__(
         :param sort_order:
             Whether to sort the meta field by ascending or descending order.
             Possible values are `descending` (default) and `ascending`.
+        :param missing_meta:
+            What to do with documents that are missing the sorting metadata field.
+            Possible values are:
+            - 'drop' will drop the documents entirely.
+            - 'top' will place the documents at the top of the metadata-sorted list
+                (regardless of 'ascending' or 'descending').
+            - 'bottom' will place the documents at the bottom of metadata-sorted list
+                (regardless of 'ascending' or 'descending').
         :param meta_value_type:
             Parse the meta value into the data type specified before sorting.
             This will only work if all meta values stored under `meta_field` in the provided documents are strings.
@@ -82,11 +91,13 @@ def __init__(
         self.top_k = top_k
         self.ranking_mode = ranking_mode
         self.sort_order = sort_order
+        self.missing_meta = missing_meta
         self._validate_params(
             weight=self.weight,
             top_k=self.top_k,
             ranking_mode=self.ranking_mode,
             sort_order=self.sort_order,
+            missing_meta=self.missing_meta,
             meta_value_type=meta_value_type,
         )
         self.meta_value_type = meta_value_type
@@ -97,6 +108,7 @@ def _validate_params(
         top_k: Optional[int],
         ranking_mode: Literal["reciprocal_rank_fusion", "linear_score"],
         sort_order: Literal["ascending", "descending"],
+        missing_meta: Literal["drop", "top", "bottom"],
         meta_value_type: Optional[Literal["float", "int", "date"]],
     ):
         if top_k is not None and top_k <= 0:
@@ -125,6 +137,14 @@ def _validate_params(
                 "MetaFieldRanker." % sort_order
             )
 
+        if missing_meta not in ["drop", "top", "bottom"]:
+            raise ValueError(
+                "The value of parameter <missing_meta> must be 'drop', 'top', or 'bottom', "
+                "but is currently set to '%s'.\n"
+                "Change the <missing_meta> value to 'drop', 'top', or 'bottom' when initializing the "
+                "MetaFieldRanker." % missing_meta
+            )
+
         if meta_value_type not in ["float", "int", "date", None]:
             raise ValueError(
                 "The value of parameter <meta_value_type> must be 'float', 'int', 'date' or None but is "
@@ -141,6 +161,7 @@ def run(
         weight: Optional[float] = None,
         ranking_mode: Optional[Literal["reciprocal_rank_fusion", "linear_score"]] = None,
         sort_order: Optional[Literal["ascending", "descending"]] = None,
+        missing_meta: Optional[Literal["drop", "top", "bottom"]] = None,
         meta_value_type: Optional[Literal["float", "int", "date"]] = None,
     ):
         """
@@ -171,6 +192,15 @@ def run(
             Whether to sort the meta field by ascending or descending order.
             Possible values are `descending` (default) and `ascending`.
             If not provided, the sort_order provided at initialization time is used.
+        :param missing_meta:
+            What to do with documents that are missing the sorting metadata field.
+            Possible values are:
+            - 'drop' will drop the documents entirely.
+            - 'top' will place the documents at the top of the metadata-sorted list
+                (regardless of 'ascending' or 'descending').
+            - 'bottom' will place the documents at the bottom of metadata-sorted list
+                (regardless of 'ascending' or 'descending').
+            If not provided, the missing_meta provided at initialization time is used.
         :param meta_value_type:
             Parse the meta value into the data type specified before sorting.
             This will only work if all meta values stored under `meta_field` in the provided documents are strings.
@@ -199,12 +229,14 @@ def run(
         weight = weight if weight is not None else self.weight
         ranking_mode = ranking_mode or self.ranking_mode
         sort_order = sort_order or self.sort_order
+        missing_meta = missing_meta or self.missing_meta
         meta_value_type = meta_value_type or self.meta_value_type
         self._validate_params(
             weight=weight,
             top_k=top_k,
             ranking_mode=ranking_mode,
             sort_order=sort_order,
+            missing_meta=missing_meta,
             meta_value_type=meta_value_type,
         )
 
@@ -227,12 +259,27 @@ def run(
             return {"documents": documents[:top_k]}
 
         if len(docs_missing_meta_field) > 0:
-            logger.warning(
-                "The parameter <meta_field> is currently set to '{meta_field}' but the Documents with IDs {document_ids} don't have this meta key.\n"
-                "These Documents will be placed at the end of the sorting order.",
-                meta_field=self.meta_field,
-                document_ids=",".join([doc.id for doc in docs_missing_meta_field]),
-            )
+            if missing_meta == "bottom":
+                logger.warning(
+                    "The parameter <meta_field> is currently set to '{meta_field}' but the Documents with IDs {document_ids} don't have this meta key.\n"
+                    "Because the parameter <missing_meta> is set to 'bottom', these Documents will be placed at the end of the sorting order.",
+                    meta_field=self.meta_field,
+                    document_ids=",".join([doc.id for doc in docs_missing_meta_field]),
+                )
+            elif missing_meta == "top":
+                logger.warning(
+                    "The parameter <meta_field> is currently set to '{meta_field}' but the Documents with IDs {document_ids} don't have this meta key.\n"
+                    "Because the parameter <missing_meta> is set to 'top', these Documents will be placed at the top of the sorting order.",
+                    meta_field=self.meta_field,
+                    document_ids=",".join([doc.id for doc in docs_missing_meta_field]),
+                )
+            else:
+                logger.warning(
+                    "The parameter <meta_field> is currently set to '{meta_field}' but the Documents with IDs {document_ids} don't have this meta key.\n"
+                    "Because the parameter <missing_meta> is set to 'drop', these Documents will be removed from the list of retrieved Documents.",
+                    meta_field=self.meta_field,
+                    document_ids=",".join([doc.id for doc in docs_missing_meta_field]),
+                )
 
         # If meta_value_type is provided try to parse the meta values
         parsed_meta = self._parse_meta(docs_with_meta_field=docs_with_meta_field, meta_value_type=meta_value_type)
@@ -252,10 +299,18 @@ def run(
             )
             return {"documents": documents[:top_k]}
 
-        # Add the docs missing the meta_field back on the end
+        # Merge rankings and handle missing meta fields as specified in the missing_meta parameter
         sorted_by_meta = [doc for meta, doc in tuple_sorted_by_meta]
-        sorted_documents = sorted_by_meta + docs_missing_meta_field
-        sorted_documents = self._merge_rankings(documents, sorted_documents, weight, ranking_mode)
+        if missing_meta == "bottom":
+            sorted_documents = sorted_by_meta + docs_missing_meta_field
+            sorted_documents = self._merge_rankings(documents, sorted_documents, weight, ranking_mode)
+        elif missing_meta == "top":
+            sorted_documents = docs_missing_meta_field + sorted_by_meta
+            sorted_documents = self._merge_rankings(documents, sorted_documents, weight, ranking_mode)
+        else:
+            sorted_documents = sorted_by_meta
+            sorted_documents = self._merge_rankings(docs_with_meta_field, sorted_documents, weight, ranking_mode)
+
         return {"documents": sorted_documents[:top_k]}
 
     def _parse_meta(
diff --git a/releasenotes/notes/metaranker-missing-meta-options-1a969008d632a523.yaml b/releasenotes/notes/metaranker-missing-meta-options-1a969008d632a523.yaml
new file mode 100644
index 0000000000..46cb0c82ab
--- /dev/null
+++ b/releasenotes/notes/metaranker-missing-meta-options-1a969008d632a523.yaml
@@ -0,0 +1,7 @@
+---
+features:
+  - |
+    Add a new `missing_meta` param to `MetaFieldRanker`, which determines what to do with
+    documents that lack the ranked meta field. Supported values are `"bottom"` (which
+    puts documents with missing meta at the bottom of the sorted list), `"top"` (which puts them
+    at the top), and `"drop"` (which removes them from the results entirely).
diff --git a/test/components/rankers/test_metafield.py b/test/components/rankers/test_metafield.py
index beb105bcf8..87366e27c5 100644
--- a/test/components/rankers/test_metafield.py
+++ b/test/components/rankers/test_metafield.py
@@ -175,6 +175,10 @@ def test_raises_value_error_if_wrong_sort_order(self):
         with pytest.raises(ValueError):
             MetaFieldRanker(meta_field="rating", sort_order="wrong_order")
 
+    def test_raises_value_error_if_wrong_missing_meta(self):
+        with pytest.raises(ValueError):
+            MetaFieldRanker(meta_field="rating", missing_meta="wrong_missing_meta")
+
     def test_raises_value_error_if_wrong_meta_value_type(self):
         with pytest.raises(ValueError):
             MetaFieldRanker(meta_field="rating", meta_value_type="wrong_type")
@@ -239,3 +243,39 @@ def test_different_ranking_mode_for_init_vs_run(self):
         output = ranker.run(documents=docs_before, ranking_mode="reciprocal_rank_fusion")
         docs_after = output["documents"]
         assert docs_after[0].score == pytest.approx(0.016261, abs=1e-5)
+
+    def test_missing_meta_bottom(self):
+        ranker = MetaFieldRanker(meta_field="rating", ranking_mode="linear_score", weight=0.5, missing_meta="bottom")
+        docs_before = [
+            Document(id="1", content="abc", meta={"rating": 1.3}, score=0.6),
+            Document(id="2", content="abc", meta={}, score=0.4),
+            Document(id="3", content="abc", meta={"rating": 2.1}, score=0.39),
+        ]
+        output = ranker.run(documents=docs_before)
+        docs_after = output["documents"]
+        assert len(docs_after) == 3
+        assert docs_after[2].id == "2"
+
+    def test_missing_meta_top(self):
+        ranker = MetaFieldRanker(meta_field="rating", ranking_mode="linear_score", weight=0.5, missing_meta="top")
+        docs_before = [
+            Document(id="1", content="abc", meta={"rating": 1.3}, score=0.6),
+            Document(id="2", content="abc", meta={}, score=0.59),
+            Document(id="3", content="abc", meta={"rating": 2.1}, score=0.4),
+        ]
+        output = ranker.run(documents=docs_before)
+        docs_after = output["documents"]
+        assert len(docs_after) == 3
+        assert docs_after[0].id == "2"
+
+    def test_missing_meta_drop(self):
+        ranker = MetaFieldRanker(meta_field="rating", ranking_mode="linear_score", weight=0.5, missing_meta="drop")
+        docs_before = [
+            Document(id="1", content="abc", meta={"rating": 1.3}, score=0.6),
+            Document(id="2", content="abc", meta={}, score=0.59),
+            Document(id="3", content="abc", meta={"rating": 2.1}, score=0.4),
+        ]
+        output = ranker.run(documents=docs_before)
+        docs_after = output["documents"]
+        assert len(docs_after) == 2
+        assert "2" not in [doc.id for doc in docs_after]


Code review comments:
diff_hunk: @@ -227,12 +259,27 @@ def run(
             return {"documents": documents[:top_k]}
 
         if len(docs_missing_meta_field) > 0:
-            logger.warning(
-                "The parameter <meta_field> is currently set to '{meta_field}' but the Documents with IDs {document_ids} don't have this meta key.\n"
-                "These Documents will be placed at the end of the sorting order.",
-                meta_field=self.meta_field,
-                document_ids=",".join([doc.id for doc in docs_missing_meta_field]),
-            )
+            if missing_meta == "bottom":
+                logger.warning(
+                    "The parameter <meta_field> is currently set to '{meta_field}' but the Documents with IDs {document_ids} don't have this meta key.\n"
path: haystack/components/rankers/meta_field.py
text: @user1:
Looks like this string can be extracted into a variable and reused in all three warnings?