Repository: deepset-ai/haystack

Issue:
Weights and score normalization for `JoinDocuments` node with reciprocal rank fusion (PR ready)
**Is your feature request related to a problem? Please describe.**
This PR would address two existing problems with using the `JoinDocuments` node with reciprocal rank fusion (RRF):
1. The node's `weights` are not applied with RRF, and input rankings are always weighted equally.
2. Scores are not normalized. As a result, the output documents receive their raw RRF scores, which are not very useful.

**Describe the solution you'd like**
I've got a PR ready that weights and normalizes the RRF scores. As it's currently implemented, the normalization would always happen--I've yet to see a use case where we'd want the raw RRF scores--but the code can easily be revised so that normalization is optional. The changes would be non-breaking in the sense that if no weights are provided (= all input nodes are weighted equally), the output documents would be identical to those currently output by the `JoinDocuments` node (_modulo_ score normalization).

**Describe alternatives you've considered**
Not really too many alternatives to this approach beyond custom nodes, as far as I can tell. There would be some small decisions with this approach, namely, is normalization applied automatically or by choice?

**Additional context**
Hybrid search pipelines often make use of `JoinDocuments` nodes with RRF, so ensuring that weights can be applied and scores are interpretable would be quite useful.



Discussion:
Hey Rob, this seems to be an interesting addition to the `JoinDocuments` component! Please go ahead and create the PR :) 
Great, thanks @bogdankostic ! I've created a draft PR [here](https://github.com/deepset-ai/haystack/pull/5704).

Resolving patch:
diff --git a/haystack/nodes/other/join_docs.py b/haystack/nodes/other/join_docs.py
--- a/haystack/nodes/other/join_docs.py
+++ b/haystack/nodes/other/join_docs.py
@@ -155,8 +155,16 @@ def _calculate_rrf(self, results):
         K = 61
 
         scores_map = defaultdict(int)
-        for result in results:
+        weights = self.weights if self.weights else [1 / len(results)] * len(results)
+
+        # Calculate weighted reciprocal rank fusion score
+        for result, weight in zip(results, weights):
             for rank, doc in enumerate(result):
-                scores_map[doc.id] += 1 / (K + rank)
+                scores_map[doc.id] += (weight * len(results)) / (K + rank)
+
+        # Normalize scores. Note: len(results) / K is the maximum possible score,
+        # achieved by being ranked first in all results with non-zero weight.
+        for id in scores_map:
+            scores_map[id] = scores_map[id] / (len(results) / K)
 
         return scores_map


Test patch:
diff --git a/test/nodes/test_join_documents.py b/test/nodes/test_join_documents.py
--- a/test/nodes/test_join_documents.py
+++ b/test/nodes/test_join_documents.py
@@ -3,6 +3,7 @@
 
 from haystack import Document
 from haystack.nodes.other.join_docs import JoinDocuments
+from copy import deepcopy
 
 
 @pytest.mark.unit
@@ -113,3 +114,38 @@ def test_joindocuments_concatenate_duplicate_docs_null_score():
     result, _ = join_docs.run(inputs)
     assert len(result["documents"]) == 3
     assert result["documents"] == expected_outputs["documents"]
+
+
+@pytest.mark.unit
+def test_joindocuments_rrf_weights():
+    """
+    Test that the reciprocal rank fusion method correctly handles weights.
+    """
+    inputs_none = [
+        {
+            "documents": [
+                Document(content="text document 1", content_type="text", score=0.2),
+                Document(content="text document 2", content_type="text", score=0.3),
+            ]
+        },
+        {
+            "documents": [
+                Document(content="text document 3", content_type="text", score=0.7),
+                Document(content="text document 4", content_type="text", score=None),
+            ]
+        },
+    ]
+
+    inputs_even = deepcopy(inputs_none)
+    inputs_uneven = deepcopy(inputs_none)
+
+    join_docs_none = JoinDocuments(join_mode="reciprocal_rank_fusion")
+    result_none, _ = join_docs_none.run(inputs_none)
+    join_docs_even = JoinDocuments(join_mode="reciprocal_rank_fusion", weights=[0.5, 0.5])
+    result_even, _ = join_docs_even.run(inputs_even)
+    join_docs_uneven = JoinDocuments(join_mode="reciprocal_rank_fusion", weights=[0.7, 0.3])
+    result_uneven, _ = join_docs_uneven.run(inputs_uneven)
+
+    assert result_none["documents"] == result_even["documents"]
+    assert result_uneven["documents"] != result_none["documents"]
+    assert result_uneven["documents"][0].score > result_none["documents"][0].score