Repository: deepset-ai/haystack

Issue:
Pipeline Batch Eval Fails with TransformersReader
**Describe the bug**
Running pipeline.batch_eval with a TransformersReader fails in line https://github.com/deepset-ai/haystack/blob/main/haystack/nodes/reader/transformers.py#L249
because `pred` variable is a list of predictions and not a single prediction as assumed in the code.

I assume that the problem is that the return value of `self.model()` is a list of list of list (for each query there are multiple documents, and for each document there are multiple answers)

**Error message**

```python
        number_of_answers_per_query = [0]
        results: Dict = {"queries": queries, "answers": [], "no_ans_gaps": []}
        for grouped_pred, grouped_inp in zip(grouped_predictions, grouped_inputs):
            # Add Document ID to predictions to be able to construct Answer objects
            for preds_for_single_doc, inp in zip(grouped_pred, grouped_inp):
                for pred in preds_for_single_doc:
                    cur_doc_id = inp.doc_id
>                   pred["doc_id"] = cur_doc_id
E                   TypeError: list indices must be integers or slices, not str

haystack/nodes/reader/transformers.py:249: TypeError
```

**Expected behavior**
Test case `test_extractive_qa_eval` in `test_eval_batch.py` runs for TransformersReader (and FarmReader) so that we can change the parametrization to:
`@pytest.mark.parametrize("reader", ["farm", "transformers"], indirect=True)`

**Additional context**
related issue #3556 

**To Reproduce**
run the `test_extractive_qa_eval` test case in test_eval_batch.py but with TransformersReader instead of FarmReader.


**FAQ Check**
- [x] Have you had a look at [our new FAQ page](https://docs.haystack.deepset.ai/docs/faq)?



Discussion:

Resolving patch:
diff --git a/haystack/nodes/reader/transformers.py b/haystack/nodes/reader/transformers.py
--- a/haystack/nodes/reader/transformers.py
+++ b/haystack/nodes/reader/transformers.py
@@ -153,7 +153,7 @@ def predict(self, query: str, documents: List[Document], top_k: Optional[int] =
             max_seq_len=self.max_seq_len,
             doc_stride=self.doc_stride,
         )
-        # Transformers gives different output dependiing on top_k_per_candidate and number of inputs
+        # Transformers gives different output depending on top_k_per_candidate and number of inputs
         if isinstance(predictions, dict):
             predictions = [[predictions]]
         elif len(inputs) == 1:
@@ -224,7 +224,7 @@ def predict_batch(
         # Transformers flattens lists of length 1. This restores the original list structure.
         if isinstance(predictions, dict):
             predictions = [[predictions]]
-        elif len(number_of_docs) == 1:
+        elif len(inputs) == 1:
             predictions = [predictions]
         else:
             predictions = [p if isinstance(p, list) else [p] for p in predictions]


Test patch:
diff --git a/test/nodes/test_reader.py b/test/nodes/test_reader.py
--- a/test/nodes/test_reader.py
+++ b/test/nodes/test_reader.py
@@ -104,6 +104,18 @@ def test_output_batch_multiple_queries_multiple_doc_lists(reader, docs):
     assert len(prediction["answers"][0]) == 5  # top-k of 5 for collection of docs
 
 
+def test_output_batch_single_query_single_nested_doc_list(reader, docs):
+    prediction = reader.predict_batch(queries=["Who lives in Berlin?"], documents=[docs], top_k=5)
+    assert prediction is not None
+    assert prediction["queries"] == ["Who lives in Berlin?"]
+    # Expected output: List of lists answers
+    assert isinstance(prediction["answers"], list)
+    assert isinstance(prediction["answers"][0], list)
+    assert isinstance(prediction["answers"][0][0], Answer)
+    assert len(prediction["answers"]) == 1  # Predictions for 1 collections of documents
+    assert len(prediction["answers"][0]) == 5  # top-k of 5 for collection of docs
+
+
 @pytest.mark.integration
 def test_no_answer_output(no_answer_reader, docs):
     no_answer_prediction = no_answer_reader.predict(query="What is the meaning of life?", documents=docs, top_k=5)
diff --git a/test/pipelines/test_eval_batch.py b/test/pipelines/test_eval_batch.py
--- a/test/pipelines/test_eval_batch.py
+++ b/test/pipelines/test_eval_batch.py
@@ -118,7 +118,7 @@ def test_summarizer_calculate_metrics(document_store_with_docs: ElasticsearchDoc
 
 @pytest.mark.parametrize("retriever_with_docs", ["tfidf"], indirect=True)
 @pytest.mark.parametrize("document_store_with_docs", ["memory"], indirect=True)
-@pytest.mark.parametrize("reader", ["farm"], indirect=True)
+@pytest.mark.parametrize("reader", ["farm", "transformers"], indirect=True)
 def test_extractive_qa_eval(reader, retriever_with_docs, tmp_path):
     labels = EVAL_LABELS[:1]