Repository: deepset-ai/haystack

Issue:
Combination of multiple retrievers
Originating from a discussion in #104. 

**What**
Combine multiple scores during retrieval (e.g. embeddings similarity of multiple text fields, or BM25 + Embedding similarity ...) 

**Options**

a) Concatenation of Retrievers: Make the Finder accept a list of Retrievers and combine their scores (incl. weights) before feeding results to the Reader
Pro: most generic solution also allowing combination of BM25 + EmbeddingRetriever
Con: Two separate retriever queries (suboptimal efficiency in some cases)

b) New MultiEmbeddingRetriever: Extend the EmbeddingRetriever to the case of multiple Embeddings
Pro: simple implementation without side effects on the regular user interface
Con: won't allow the combination of other retriever methods (e.g. ElasticsearchRetriever + EmbeddingRetriver)

I am leaning towards a), but could see b) as a simpler short-term solution for the original use case discussed in #104 .



┆Issue is synchronized with this [Jira Task](https://deepset-ai.atlassian.net/browse/HS-42) by [Unito](https://www.unito.io/learn-more)



Discussion:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs.

Still a useful feature. While nothing immediate on our roadmap I would still opt for keeping this one open.
Hey @tholor , I believe this could be a great feature. Especially since BM25 cost almost nothing, and can sometimes bring more suitable documents than EmbeddingRetriver or DensePassageRetriever. We can imagine retrieving k texts for each Retriever, and letting the reader rank all answers afterwards.
Yes, totally agree @guillim. It's on our roadmap for later this year / Q1 2021. 
While it's quite straightforward from a technical perspective, I think we need to put some thoughts into the design of the user-facing API (should Finder get a list of retrievers or maybe becomes a fully flexible pipeline where you just stack retrievers, readers, rerankers ...). It will also become tricky in case we support the combination of retrievers that use different documentstores (e.g. DPR with FAISS + Elastic's BM25 with Elasticsearch)  
@guillim  Quick update:  We have the first version now implemented and you can use it like this: 

```python
from haystack.pipeline import DocumentSearchPipeline, ExtractiveQAPipeline, Pipeline, JoinDocuments
from haystack.document_store.elasticsearch import ElasticsearchDocumentStore
from haystack.reader.farm import FARMReader
from haystack.retriever.sparse import ElasticsearchRetriever

# Building blocks (= Nodes)
document_store = ElasticsearchDocumentStore(host="localhost", username="", password="", index="document")
es_retriever = ElasticsearchRetriever(document_store=document_store)
reader = FARMReader(model_name_or_path="deepset/roberta-base-squad2", use_gpu=False)
...

# Combine via default pipeline (here: QA)
qa_pipe = ExtractiveQAPipeline(reader=reader, retriever=retriever)
res = qa_pipe.run(question="Who is the father of Sansa Stark?", top_k_retriever=2, top_k_reader=5)
print(res)
qa_pipe.draw()


# Or build your own custom pipeline to model complex search routes (e.g. combine multiple retrievers)
p = Pipeline()
p.add_node(component=es_retriever, name="ESRetriever1", inputs=["Query"])
p.add_node(component=dpr_retriever, name="DPRRetriever1", inputs=["Query"])
p.add_node(component=JoinDocuments(join_mode="concatenate"), name="JoinResults", inputs=["ESRetriever1", "DPRRetriever1"])
```

@tanaysoni We still need to add other options for `join_mode` (e.g. combining doc scores + weighting)
Nice ! I'll give it a shot very soon.

Resolving patch:
diff --git a/haystack/pipeline.py b/haystack/pipeline.py
--- a/haystack/pipeline.py
+++ b/haystack/pipeline.py
@@ -1,3 +1,4 @@
+from copy import deepcopy
 from pathlib import Path
 from typing import List, Optional, Dict
 
@@ -280,19 +281,63 @@ def run(self, **kwargs):
 
 
 class JoinDocuments:
+    """
+    A node to join documents outputted by multiple retriever nodes.
+
+    The node allows multiple join modes:
+    * concatenate: combine the documents from multiple nodes. Any duplicate documents are discarded.
+    * merge: merge scores of documents from multiple nodes. Optionally, each input score can be given a different
+             `weight` & a `top_k` limit can be set. This mode can also be used for "reranking" retrieved documents.
+    """
+
     outgoing_edges = 1
 
-    def __init__(self, join_mode="concatenate"):
-        pass
+    def __init__(
+        self, join_mode: str = "concatenate", weights: Optional[List[float]] = None, top_k_join: Optional[int] = None
+    ):
+        """
+        :param join_mode: `concatenate` to combine documents from multiple retrievers or `merge` to aggregate scores of
+                          individual documents.
+        :param weights: A node-wise list(length of list must be equal to the number of input nodes) of weights for
+                        adjusting document scores when using the `merge` join_mode. By default, equal weight is given
+                        to each retriever score. This param is not compatible with the `concatenate` join_mode.
+        :param top_k_join: Limit documents to top_k based on the resulting scores of the join.
+        """
+        assert join_mode in ["concatenate", "merge"], f"JoinDocuments node does not support '{join_mode}' join_mode."
+
+        assert not (
+            weights is not None and join_mode == "concatenate"
+        ), "Weights are not compatible with 'concatenate' join_mode."
+        self.join_mode = join_mode
+        self.weights = weights
+        self.top_k = top_k_join
 
     def run(self, **kwargs):
         inputs = kwargs["inputs"]
 
-        documents = []
-        for i, _ in inputs:
-            documents.extend(i["documents"])
-        output = {
-            "query": inputs[0][0]["query"],
-            "documents": documents
-        }
+        if self.join_mode == "concatenate":
+            document_map = {}
+            for input_from_node, _ in inputs:
+                for doc in input_from_node["documents"]:
+                    document_map[doc.id] = doc
+        elif self.join_mode == "merge":
+            document_map = {}
+            if self.weights:
+                weights = self.weights
+            else:
+                weights = [1/len(inputs)] * len(inputs)
+            for (input_from_node, _), weight in zip(inputs, weights):
+                for doc in input_from_node["documents"]:
+                    if document_map.get(doc.id):  # document already exists; update score
+                        document_map[doc.id].score += doc.score * weight
+                    else:  # add the document in map
+                        document_map[doc.id] = deepcopy(doc)
+                        document_map[doc.id].score *= weight
+        else:
+            raise Exception(f"Invalid join_mode: {self.join_mode}")
+
+        documents = sorted(document_map.values(), key=lambda d: d.score, reverse=True)
+        if self.top_k:
+            documents = documents[: self.top_k]
+        output = {"query": inputs[0][0]["query"], "documents": documents}
         return output, "output_1"


Test patch:
diff --git a/test/test_pipeline.py b/test/test_pipeline.py
--- a/test/test_pipeline.py
+++ b/test/test_pipeline.py
@@ -1,7 +1,9 @@
 import pytest
 
 from haystack.document_store.elasticsearch import ElasticsearchDocumentStore
-from haystack.pipeline import ExtractiveQAPipeline, Pipeline, FAQPipeline, DocumentSearchPipeline
+from haystack.pipeline import JoinDocuments, ExtractiveQAPipeline, Pipeline, FAQPipeline, DocumentSearchPipeline
+from haystack.retriever.dense import DensePassageRetriever
+from haystack.retriever.sparse import ElasticsearchRetriever
 
 
 @pytest.mark.slow
@@ -117,3 +119,56 @@ def test_document_search_pipeline(retriever, document_store):
     if isinstance(document_store, ElasticsearchDocumentStore):
         output = pipeline.run(query="How to test this?", filters={"source": ["wiki2"]}, top_k_retriever=5)
         assert len(output["documents"]) == 1
+
+
+@pytest.mark.parametrize("document_store_with_docs", ["elasticsearch"], indirect=True)
+@pytest.mark.parametrize("reader", ["farm"], indirect=True)
+def test_join_document_pipeline(document_store_with_docs, reader):
+    es = ElasticsearchRetriever(document_store=document_store_with_docs)
+    dpr = DensePassageRetriever(
+        document_store=document_store_with_docs,
+        query_embedding_model="facebook/dpr-question_encoder-single-nq-base",
+        passage_embedding_model="facebook/dpr-ctx_encoder-single-nq-base",
+        use_gpu=False,
+    )
+    document_store_with_docs.update_embeddings(dpr)
+
+    query = "Where does Carla lives?"
+
+    # test merge without weights
+    join_node = JoinDocuments(join_mode="merge")
+    p = Pipeline()
+    p.add_node(component=es, name="R1", inputs=["Query"])
+    p.add_node(component=dpr, name="R2", inputs=["Query"])
+    p.add_node(component=join_node, name="Join", inputs=["R1", "R2"])
+    results = p.run(query=query)
+    assert len(results["documents"]) == 3
+
+    # test merge with weights
+    join_node = JoinDocuments(join_mode="merge", weights=[1000, 1], top_k_join=2)
+    p = Pipeline()
+    p.add_node(component=es, name="R1", inputs=["Query"])
+    p.add_node(component=dpr, name="R2", inputs=["Query"])
+    p.add_node(component=join_node, name="Join", inputs=["R1", "R2"])
+    results = p.run(query=query)
+    assert results["documents"][0].score > 1000
+    assert len(results["documents"]) == 2
+
+    # test concatenate
+    join_node = JoinDocuments(join_mode="concatenate")
+    p = Pipeline()
+    p.add_node(component=es, name="R1", inputs=["Query"])
+    p.add_node(component=dpr, name="R2", inputs=["Query"])
+    p.add_node(component=join_node, name="Join", inputs=["R1", "R2"])
+    results = p.run(query=query)
+    assert len(results["documents"]) == 3
+
+    # test join_node with reader
+    join_node = JoinDocuments()
+    p = Pipeline()
+    p.add_node(component=es, name="R1", inputs=["Query"])
+    p.add_node(component=dpr, name="R2", inputs=["Query"])
+    p.add_node(component=join_node, name="Join", inputs=["R1", "R2"])
+    p.add_node(component=reader, name="Reader", inputs=["Join"])
+    results = p.run(query=query)
+    assert results["answers"][0]["answer"] == "Berlin"