Repository: deepset-ai/haystack

Issue:
Unable to use Generator instead of Reader in Text and Table QA Pipeline
**Describe the bug**
I used generator in the colab file which demonstrates the capability of haystack pipeline to use both text and table qa together.

But I get an error because of the join answer node in the pipeline

**Error message**
Error that was thrown (if available)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
[/usr/local/lib/python3.7/dist-packages/haystack/pipelines/base.py](https://localhost:8080/#) in run(self, query, file_paths, labels, documents, meta, params, debug)
    651                     logger.debug(f"Running node `{node_id}` with input `{node_input}`")
--> 652                     node_output, stream_id = self.graph.nodes[node_id]["component"]._dispatch_run(**node_input)
    653                 except Exception as e:

4 frames
TypeError: '<' not supported between instances of 'NoneType' and 'float'

During handling of the above exception, another exception occurred:

Exception                                 Traceback (most recent call last)
[/usr/local/lib/python3.7/dist-packages/haystack/pipelines/base.py](https://localhost:8080/#) in run(self, query, file_paths, labels, documents, meta, params, debug)
    654                     tb = traceback.format_exc()
    655                     raise Exception(
--> 656                         f"Exception while running node `{node_id}` with input `{node_input}`: {e}, full stack trace: {tb}"
    657                     )
    658                 queue.pop(node_id)

Exception: Exception while running node `JoinAnswers` with input `{'inputs': [{'query': 'Who was Thomas Alva Edison?', 'answers': [<Answer {'answer': "Thomas Alva Edison (February 11, 1847October 18, 1931) was an American inventor and businessman who has been described as America's greatest inventor. He is credited with inventing many devices in fields such as electric power generation, mass communication, sound recording, motion picture camera, and the long-lasting, practical electric light bulb. He also founded the American Telephone and Telegraph Company (AT&T) in 1885.", 'type': 'generative', 'score': None, 'context': None, 'offsets_in_document': None, 'offsets_in_context': None, 'document_id': None, 'meta': {'doc_ids': ['8f14e58c-1ad4-4de3-b1f4-eb69bc62e742', '08306284-42f6-407e-b1d8-e1ada1bf0f6d', '6ac15758-97b3-40ce-ba52-33ea927d395c', 'e6a91a97-3b9c-4617-8809-048f39a28398'], 'doc_scores': [0.8367179550744909, 0.816386816304066, 0.809023410230337, 0.8086543358442457], 'content': ["Thomas Edison Thomas Alva Edison (February 11, 1847October 18, 1931) was an American inventor and businessman, who has been described as America's greatest inventor. He is credited with developing many devices in fields such as electric power generation, mass communication, sound recording, and motion pictures. These inventions, which include the phonograph, the motion picture camera, and the long-lasting, practical electric light bulb, had a widespread impact on the modern indu...
0   1928      Alan Titherley          R. Finch   
1   1929          T. P. Dick  Margaret Tragett   
2   1930         Donald Hume   Dorothy Colpoys   
3   1931          T. P. Dick   Dorothy Colpoys   
4   1932         W. Hamilton  Thelma Kingsbury   
..   ...                 ...               ...

**Expected behavior**
A clear and concise description of what you expected to happen.

**Additional context**
Add any other context about the problem here, like document types / preprocessing steps / settings of reader etc.

**To Reproduce**
Steps to reproduce the behavior

**FAQ Check**
- [ ] Have you had a look at [our new FAQ page](https://haystack.deepset.ai/overview/faq)?

**System:**
 - OS: 
 - GPU/CPU:
 - Haystack version (commit or version number): 
 - DocumentStore:
 - Reader:
 - Retriever:



Discussion:

Resolving patch:
diff --git a/haystack/nodes/other/join_answers.py b/haystack/nodes/other/join_answers.py
--- a/haystack/nodes/other/join_answers.py
+++ b/haystack/nodes/other/join_answers.py
@@ -10,7 +10,11 @@ class JoinAnswers(BaseComponent):
     """
 
     def __init__(
-        self, join_mode: str = "concatenate", weights: Optional[List[float]] = None, top_k_join: Optional[int] = None
+        self,
+        join_mode: str = "concatenate",
+        weights: Optional[List[float]] = None,
+        top_k_join: Optional[int] = None,
+        sort_by_score: bool = True,
     ):
         """
         :param join_mode: `"concatenate"` to combine documents from multiple `Reader`s. `"merge"` to aggregate scores
@@ -19,6 +23,9 @@ def __init__(
             adjusting `Answer` scores when using the `"merge"` join_mode. By default, equal weight is assigned to each
             `Reader` score. This parameter is not compatible with the `"concatenate"` join_mode.
         :param top_k_join: Limit `Answer`s to top_k based on the resulting scored of the join.
+        :param sort_by_score: Whether to sort the incoming answers by their score. Set this to True if your Answers
+            are coming from a Reader or TableReader. Set to False if any Answers come from a Generator since this assigns
+            None as a score to each.
         """
 
         assert join_mode in ["concatenate", "merge"], f"JoinAnswers node does not support '{join_mode}' join_mode."
@@ -31,6 +38,7 @@ def __init__(
         self.join_mode = join_mode
         self.weights = [float(i) / sum(weights) for i in weights] if weights else None
         self.top_k_join = top_k_join
+        self.sort_by_score = sort_by_score
 
     def run(self, inputs: List[Dict], top_k_join: Optional[int] = None) -> Tuple[Dict, str]:  # type: ignore
         reader_results = [inp["answers"] for inp in inputs]
@@ -40,12 +48,13 @@ def run(self, inputs: List[Dict], top_k_join: Optional[int] = None) -> Tuple[Dic
 
         if self.join_mode == "concatenate":
             concatenated_answers = [answer for cur_reader_result in reader_results for answer in cur_reader_result]
-            concatenated_answers = sorted(concatenated_answers, reverse=True)[:top_k_join]
+            if self.sort_by_score:
+                concatenated_answers = sorted(concatenated_answers, reverse=True)
+            concatenated_answers = concatenated_answers[:top_k_join]
             return {"answers": concatenated_answers, "labels": inputs[0].get("labels", None)}, "output_1"
 
         elif self.join_mode == "merge":
             merged_answers = self._merge_answers(reader_results)
-
             merged_answers = merged_answers[:top_k_join]
             return {"answers": merged_answers, "labels": inputs[0].get("labels", None)}, "output_1"
 
@@ -59,5 +68,8 @@ def _merge_answers(self, reader_results: List[List[Answer]]) -> List[Answer]:
             for answer in result:
                 if isinstance(answer.score, float):
                     answer.score *= weight
+        merged_answers = [answer for cur_reader_result in reader_results for answer in cur_reader_result]
+        if self.sort_by_score:
+            merged_answers = sorted(merged_answers, reverse=True)
 
         return sorted([answer for cur_reader_result in reader_results for answer in cur_reader_result], reverse=True)


Test patch: