Repository: deepset-ai/haystack

Issue:
Tutorial 5 fails on reader.eval() call
**Describe the bug**
The google colab version of [tutorial 5 (evaluation)](https://colab.research.google.com/github/deepset-ai/haystack/blob/master/tutorials/Tutorial5_Evaluation.ipynb) fails on executing the line:
`reader_eval_results = reader.eval(document_store=document_store, device=device, label_index=label_index, doc_index=doc_index)` with the following type error:

**Error message**
```
/usr/local/lib/python3.7/dist-packages/haystack/modeling/evaluation/eval.py in <dictcomp>(.0)
     57             tqdm(self.data_loader, desc="Evaluating", mininterval=10)
     58         ):
---> 59             batch = {key: batch[key].to(self.device) for key in batch}
     60 
     61             with torch.no_grad():

TypeError: to() received an invalid combination of arguments - got (list), but expected one of:
 * (torch.device device, torch.dtype dtype, bool non_blocking, bool copy, *, torch.memory_format memory_format)
 * (torch.dtype dtype, bool non_blocking, bool copy, *, torch.memory_format memory_format)
 * (Tensor tensor, bool non_blocking, bool copy, *, torch.memory_format memory_format)
```

@brandenchan do you maybe have an idea what's the problem here?


Discussion:
It seems that I introduced this bug in #1683. It should be fixed as soon as #1696 is merged.
Oh, okay. That's good news. 👍 
Merged, should be solved! :+1: 

Resolving patch:
diff --git a/haystack/modeling/utils.py b/haystack/modeling/utils.py
--- a/haystack/modeling/utils.py
+++ b/haystack/modeling/utils.py
@@ -1,4 +1,4 @@
-from typing import Any, Iterator, Tuple
+from typing import Any, Iterator, Tuple, List
 
 import logging
 import os
@@ -48,7 +48,7 @@ def set_all_seeds(seed: int, deterministic_cudnn: bool=False) -> None:
         torch.backends.cudnn.benchmark = False
 
 
-def initialize_device_settings(use_cuda: bool, local_rank: int = -1, multi_gpu: bool = True):
+def initialize_device_settings(use_cuda: bool, local_rank: int = -1, multi_gpu: bool = True) -> Tuple[List[torch.device], int]:
     """
     Returns a list of available devices.
 
diff --git a/tutorials/Tutorial5_Evaluation.py b/tutorials/Tutorial5_Evaluation.py
--- a/tutorials/Tutorial5_Evaluation.py
+++ b/tutorials/Tutorial5_Evaluation.py
@@ -40,7 +40,7 @@ def tutorial5_evaluation():
     # Code
     ##############################################
     launch_es()
-    device, n_gpu = initialize_device_settings(use_cuda=True)
+    devices, n_gpu = initialize_device_settings(use_cuda=True)
 
     # Download evaluation data, which is a subset of Natural Questions development set containing 50 documents
     doc_dir = "../data/nq"
@@ -109,7 +109,7 @@ def tutorial5_evaluation():
     eval_retriever = EvalDocuments()
     eval_reader = EvalAnswers(sas_model="sentence-transformers/paraphrase-multilingual-mpnet-base-v2")
 
-    ## Evaluate Retriever on its own in closed domain fashion
+    # Evaluate Retriever on its own in closed domain fashion
     if style == "retriever_closed":
         retriever_eval_results = retriever.eval(top_k=10, label_index=label_index, doc_index=doc_index)
         ## Retriever Recall is the proportion of questions for which the correct document containing the answer is
@@ -120,7 +120,7 @@ def tutorial5_evaluation():
 
     # Evaluate Reader on its own in closed domain fashion (i.e. SQuAD style)
     elif style == "reader_closed":
-        reader_eval_results = reader.eval(document_store=document_store, device=device, label_index=label_index, doc_index=doc_index)
+        reader_eval_results = reader.eval(document_store=document_store, device=devices[0], label_index=label_index, doc_index=doc_index)
         # Evaluation of Reader can also be done directly on a SQuAD-formatted file without passing the data to Elasticsearch
         #reader_eval_results = reader.eval_on_file("../data/nq", "nq_dev_subset_v2.json", device=device)
 


Test patch: