internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: deepset-ai/haystack

Issue:
About FARMReader method predict_batch
**Question**
About method 
`FARMReader.predict_batch(query_doc_list: List[dict], top_k: int = None, batch_size: int = None)`


My Code: 
```
# Label, Document are from haystack.schema
# documents: List[Document], length of documents is 10, cuz I set top_k_retriever => 10
query_doc_list = []
for-loop:
  # some operations
  query_doc_list.append({
              "question": Label(question=question, answer=ground_truth, is_correct_answer=False,
                                is_correct_document=False, origin="web-feedback"),
              "docs": documents
  })
results = model.predict_batch(query_doc_list, top_k_reader, batch_size=batch_size)
```

I try to boost this call by increasing the batch_size[8 -> 12 -> 50 -> 100], but no matter how I modify this argument, the speed remains the same and the memory used in GPU remains the same. Seems like this argument wasn't taken into account. 

Could you please help me with this? I dunno if I called it wrong or is this a bug?



Discussion:

Resolving patch:
diff --git a/haystack/reader/farm.py b/haystack/reader/farm.py
--- a/haystack/reader/farm.py
+++ b/haystack/reader/farm.py
@@ -309,7 +309,7 @@ def predict_batch(self, query_doc_list: List[dict], top_k: int = None, batch_siz
         self.inferencer.batch_size = batch_size
         # make predictions on all document-query pairs
         predictions = self.inferencer.inference_from_objects(
-            objects=inputs, return_json=False, multiprocessing_chunksize=1
+            objects=inputs, return_json=False, multiprocessing_chunksize=10
         )
 
         # group predictions together


Test patch:
Repository: deepset-ai/haystack

Issue:
TypeError: unsupported operand type(s) for /: 'str' and 'str'
I know this related to farm, however this is triggered in haystack.tutorials.DPR_training

**Describe the bug**

training DensePassageRetriever


```
--> 220             train_file = self.processor.data_dir / self.processor.train_filename
    221             logger.info("Loading train set from: {} ".format(train_file))
    222             self.data["train"], self.tensor_names = self._get_dataset(train_file)

TypeError: unsupported operand type(s) for /: 'str' and 'str'
```
don't you think `os.path.join` would be simpler to join dir and file


**To Reproduce**
tutorial dpr training notebook on colab


**System:**
 - environment: colab
 - Python 3.7.10
 - Haystack installed by `!pip install git+https://github.com/deepset-ai/haystack.git`
 - Retriever: DensePassageRetriever



Discussion:
@tholor could you please check out this one!

Resolving patch:
diff --git a/haystack/retriever/dense.py b/haystack/retriever/dense.py
--- a/haystack/retriever/dense.py
+++ b/haystack/retriever/dense.py
@@ -280,6 +280,7 @@ def train(self,
               train_filename: str,
               dev_filename: str = None,
               test_filename: str = None,
+              max_sample: int = None,
               max_processes: int = 128,
               dev_split: float = 0,
               batch_size: int = 2,
@@ -306,6 +307,7 @@ def train(self,
         :param train_filename: training filename
         :param dev_filename: development set filename, file to be used by model in eval step of training
         :param test_filename: test set filename, file to be used by model in test step after training
+        :param max_sample: maximum number of input samples to convert. Can be used for debugging a smaller dataset.
         :param max_processes: the maximum number of processes to spawn in the multiprocessing.Pool used in DataSilo.
                               It can be set to 1 to disable the use of multiprocessing or make debugging easier.
         :param dev_split: The proportion of the train set that will sliced. Only works if dev_filename is set to None
@@ -329,10 +331,11 @@ def train(self,
         """
 
         self.processor.embed_title = embed_title
-        self.processor.data_dir = data_dir
+        self.processor.data_dir = Path(data_dir)
         self.processor.train_filename = train_filename
         self.processor.dev_filename = dev_filename
         self.processor.test_filename = test_filename
+        self.processor.max_sample = max_sample
         self.processor.dev_split = dev_split
         self.processor.num_hard_negatives = num_hard_negatives
         self.processor.num_positives = num_positives


Test patch:
Repository: deepset-ai/haystack

Issue:
Add option to filter out similar answers from results
**Is your feature request related to a problem? Please describe.**
In extractive QA, a model sometimes returns several highly similar answers (e.g. "Eiffeltower" and "the Eiffeltower").
We should give an option to filter out those similar ones to have more variety in the top_k answers.

**Describe the solution you'd like**
We already have an option for it in FARM for filtering based in positional overlap of answers (https://github.com/deepset-ai/FARM/pull/680)
Let's make the `duplicate_filtering` parameter available in the `FARMReader`. 




Discussion:

Resolving patch:
diff --git a/haystack/reader/farm.py b/haystack/reader/farm.py
--- a/haystack/reader/farm.py
+++ b/haystack/reader/farm.py
@@ -53,7 +53,8 @@ def __init__(
         num_processes: Optional[int] = None,
         max_seq_len: int = 256,
         doc_stride: int = 128,
-        progress_bar: bool = True
+        progress_bar: bool = True,
+        duplicate_filtering: int = 0
     ):
 
         """
@@ -91,6 +92,8 @@ def __init__(
         :param doc_stride: Length of striding window for splitting long texts (used if ``len(text) > max_seq_len``)
         :param progress_bar: Whether to show a tqdm progress bar or not.
                              Can be helpful to disable in production deployments to keep the logs clean.
+        :param duplicate_filtering: Answers are filtered based on their position. Both start and end position of the answers are considered.
+                                    The higher the value, answers that are more apart are filtered out. 0 corresponds to exact duplicates. -1 turns off duplicate removal.
         """
 
         # save init parameters to enable export of component config as YAML
@@ -99,6 +102,7 @@ def __init__(
             batch_size=batch_size, use_gpu=use_gpu, no_ans_boost=no_ans_boost, return_no_answer=return_no_answer,
             top_k=top_k, top_k_per_candidate=top_k_per_candidate, top_k_per_sample=top_k_per_sample,
             num_processes=num_processes, max_seq_len=max_seq_len, doc_stride=doc_stride, progress_bar=progress_bar,
+            duplicate_filtering=duplicate_filtering
         )
 
         self.return_no_answers = return_no_answer
@@ -116,6 +120,10 @@ def __init__(
             self.inferencer.model.prediction_heads[0].n_best_per_sample = top_k_per_sample
         except:
             logger.warning("Could not set `top_k_per_sample` in FARM. Please update FARM version.")
+        try:
+            self.inferencer.model.prediction_heads[0].duplicate_filtering = duplicate_filtering
+        except:
+            logger.warning("Could not set `duplicate_filtering` in FARM. Please update FARM version.")
         self.max_seq_len = max_seq_len
         self.use_gpu = use_gpu
         self.progress_bar = progress_bar


Test patch:
Repository: deepset-ai/haystack

Issue:
Pip install from requirements fails for python 3.6
**Describe the bug**
I try to install haystack 0.8.0 dependencies from requirements.txt
According to setup.py it's compatible with python 3.6
https://github.com/deepset-ai/haystack/blob/a06e4450d1f039a48d06ca6b9345d50fbc0fbf79/setup.py#L63

However pip during installing this package:
https://github.com/deepset-ai/haystack/blob/a06e4450d1f039a48d06ca6b9345d50fbc0fbf79/requirements.txt#L20

Tries to install `faiss-cpu==1.7.0` and it depends on latest numpy that works only with python 3.7+

So probably increase minimum python version required to 3.7 or specify faiss-cpu requirements as  `faiss-cpu~=1.6.3` or something like that? 

**Error message**

```
  Collecting faiss-cpu>=1.6.3 (from -r requirements.txt (line 20))
    Downloading https://files.pythonhosted.org/packages/59/54/b4d5e08b558aca33e5ec5ad51bdf4ab722116700c48074e2560426fb1bdd/faiss-cpu-1.7.0.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "/home/drell/Projects/CrowleyMarine/haystack/venv/lib/python3.6/site-packages/setuptools-40.8.0-py3.6.egg/setuptools/sandbox.py", line 154, in save_modules
      File "/home/drell/Projects/CrowleyMarine/haystack/venv/lib/python3.6/site-packages/setuptools-40.8.0-py3.6.egg/setuptools/sandbox.py", line 195, in setup_context
      File "/home/drell/Projects/CrowleyMarine/haystack/venv/lib/python3.6/site-packages/setuptools-40.8.0-py3.6.egg/setuptools/sandbox.py", line 250, in run_setup
      File "/home/drell/Projects/CrowleyMarine/haystack/venv/lib/python3.6/site-packages/setuptools-40.8.0-py3.6.egg/setuptools/sandbox.py", line 45, in _execfile
      File "/tmp/easy_install-v3rk_bwv/numpy-1.20.3/setup.py", line 30, in <module>
        Helper class to determine the numpy include path.
    RuntimeError: Python version >= 3.7 required.
```    

**Expected behavior**
All requirements are installed succefully :)


**To Reproduce**
Just try `pip install farm-haystack` with python 3.6

**System:**
 - OS: Ubuntu 18.04.5 LTS
 - Haystack version (commit or version number): 0.8.0




Discussion:

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -60,7 +60,7 @@ def get_dependency_links(filename):
     packages=find_packages(exclude=["*.tests", "*.tests.*", "tests.*", "tests"]),
     dependency_links=dependency_links,
     install_requires=parsed_requirements,
-    python_requires=">=3.6.0",
+    python_requires=">=3.7.0",
     tests_require=["pytest"],
     classifiers=[
         "Intended Audience :: Science/Research",


Test patch:
Repository: deepset-ai/haystack

Issue:
add haystack.__version__
Lets add a version tag to Haystack : )

Similar to https://github.com/deepset-ai/FARM/pull/761



Discussion:
Can I work on this?
@bhadreshpsavani Yes you're welcome to! Let us know if you need any tips / pointers!

Resolving patch:
diff --git a/haystack/__init__.py b/haystack/__init__.py
--- a/haystack/__init__.py
+++ b/haystack/__init__.py
@@ -4,6 +4,7 @@
 from haystack.schema import Document, Label, MultiLabel, BaseComponent
 from haystack.finder import Finder
 from haystack.pipeline import Pipeline
+from haystack._version import __version__
 
 pd.options.display.max_colwidth = 80
 
diff --git a/haystack/_version.py b/haystack/_version.py
new file mode 100644
--- /dev/null
+++ b/haystack/_version.py
@@ -0,0 +1 @@
+__version__ = "0.9.0"
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -1,3 +1,5 @@
+import os
+import re
 from io import open
 
 from setuptools import find_packages, setup
@@ -45,9 +47,24 @@ def get_dependency_links(filename):
 dependency_links = get_dependency_links('requirements.txt')
 parsed_requirements = parse_requirements('requirements.txt')
 
+
+def versionfromfile(*filepath):
+    infile = os.path.join(*filepath)
+    with open(infile) as fp:
+        version_match = re.search(
+                r"^__version__\s*=\s*['\"]([^'\"]*)['\"]", fp.read(), re.M
+        )
+        if version_match:
+            return version_match.group(1)
+        raise RuntimeError("Unable to find version string in {}.".format(infile))
+
+
+here = os.path.abspath(os.path.dirname(__file__))
+_version: str = versionfromfile(here, "haystack", "_version.py")
+
 setup(
     name="farm-haystack",
-    version="0.9.0",
+    version=_version,
     author="Malte Pietsch, Timo Moeller, Branden Chan, Tanay Soni",
     author_email="malte.pietsch@deepset.ai",
     description="Neural Question Answering & Semantic Search at Scale. Use modern transformer based models like BERT to find answers in large document collections",
@@ -56,7 +73,7 @@ def get_dependency_links(filename):
     keywords="QA Question-Answering Reader Retriever semantic-search search BERT roberta albert squad mrc transfer-learning language-model transformer",
     license="Apache",
     url="https://github.com/deepset-ai/haystack",
-    download_url="https://github.com/deepset-ai/haystack/archive/0.9.0.tar.gz",
+    download_url=f"https://github.com/deepset-ai/haystack/archive/{_version}.tar.gz",
     packages=find_packages(exclude=["*.tests", "*.tests.*", "tests.*", "tests"]),
     dependency_links=dependency_links,
     install_requires=parsed_requirements,


Test patch:
Repository: deepset-ai/haystack

Issue:
Crawler does not write JSON, but serializes the result dict to a string written in a .json file
First of all, great work on Haystack! It’s an incredible library and I really enjoy playing around with it!

I noticed an odd behavior of the Crawler as in this example:

```python
from haystack.connector import Crawler

crawler = Crawler()
# crawl Haystack docs, i.e. all pages that include haystack.deepset.ai/docs/
docs = crawler.crawl(urls=["https://haystack.deepset.ai/docs/latest/get_startedmd"],
    output_dir="crawled_files",
    filter_urls= ["haystack\.deepset\.ai\/docs\/"])
```

The resulting file looks like this:

```
{'meta': {'url': 'https://haystack.deepset.ai/docs/latest/get_startedmd'}, 'text': 'Knowledge...
```

which is not valid JSON, as described in the docs. The Crawler rather simply serializes the data dict to string and writes it into a JSON file.

A working import to load the result in a next step looks like this:

```python
import ast

# docs[0] being the first result from the crawl run
with open(docs[0], 'r', encoding='utf-8') as f:
    result = ast.literal_eval(f.read())
```

instead of `json.read( ... )`.

As I don’t have the overview of the entire lib and how the created text files are used across the different pipelines, I am hesitant to use propose a solution. So I am raising this as a slightly odd behavior for now. Happy to provide a fix though given guidance from other developers.


Discussion:
In addition, it would be a great addition to work with the results directly, e.g. from a depth=0 crawl, without writing files to disk.

Resolving patch:
diff --git a/haystack/connector/crawler.py b/haystack/connector/crawler.py
--- a/haystack/connector/crawler.py
+++ b/haystack/connector/crawler.py
@@ -1,3 +1,4 @@
+import json
 import logging
 import re
 from pathlib import Path
@@ -15,10 +16,9 @@ class Crawler(BaseComponent):
     ```python
     |    from haystack.connector import Crawler
     |
-    |    crawler = Crawler()
+    |    crawler = Crawler(output_dir="crawled_files")
     |    # crawl Haystack docs, i.e. all pages that include haystack.deepset.ai/docs/
     |    docs = crawler.crawl(urls=["https://haystack.deepset.ai/docs/latest/get_startedmd"],
-    |                         output_dir="crawled_files",
     |                         filter_urls= ["haystack\.deepset\.ai\/docs\/"])
     ```
     """
@@ -143,7 +143,7 @@ def _write_to_files(self, urls: List[str], output_dir: Path, base_url: str = Non
                 data['meta']['base_url'] = base_url
             data['text'] = text
             with open(file_path, 'w', encoding='utf-8') as f:
-                f.write(str(data))
+                json.dump(data, f)
             paths.append(file_path)
 
         return paths


Test patch:
Repository: deepset-ai/haystack

Issue:
change naming of vars in tutorials
Tutorial 2
Change name of “train_data” variable to “data_dir”

Tutorial 5 in python and colab
Change parameter top_k_per_candidate into top_k



Discussion:

Resolving patch:
diff --git a/tutorials/Tutorial2_Finetune_a_model_on_your_data.py b/tutorials/Tutorial2_Finetune_a_model_on_your_data.py
--- a/tutorials/Tutorial2_Finetune_a_model_on_your_data.py
+++ b/tutorials/Tutorial2_Finetune_a_model_on_your_data.py
@@ -36,9 +36,9 @@ def tutorial2_finetune_a_model_on_your_data():
     #**Recommendation: Run training on a GPU. To do so change the `use_gpu` arguments below to `True`
 
     reader = FARMReader(model_name_or_path="distilbert-base-uncased-distilled-squad", use_gpu=True)
-    train_data = "data/squad20"
-    # train_data = "PATH/TO_YOUR/TRAIN_DATA"
-    reader.train(data_dir=train_data, train_filename="dev-v2.0.json", use_gpu=True, n_epochs=1, save_dir="my_model")
+    data_dir = "data/squad20"
+    # data_dir = "PATH/TO_YOUR/TRAIN_DATA"
+    reader.train(data_dir=data_dir, train_filename="dev-v2.0.json", use_gpu=True, n_epochs=1, save_dir="my_model")
 
     # Saving the model happens automatically at the end of training into the `save_dir` you specified
     # However, you could also save a reader manually again via:
diff --git a/tutorials/Tutorial5_Evaluation.py b/tutorials/Tutorial5_Evaluation.py
--- a/tutorials/Tutorial5_Evaluation.py
+++ b/tutorials/Tutorial5_Evaluation.py
@@ -93,7 +93,7 @@ def tutorial5_evaluation():
     # Initialize Reader
     reader = FARMReader(
         model_name_or_path="deepset/roberta-base-squad2",
-        top_k_per_candidate=4,
+        top_k=4,
         return_no_answer=True
     )
 


Test patch:
Repository: deepset-ai/haystack

Issue:
Catch Elastic's search_phase_execution_exception (dense retrieval if not all documents have an embedding)
**Describe the bug**
When you run dense retrieval (via `query_by_embedding`) in ElasticsearchDocumentStore without running `update_embeddings()` before you get an exception as not all documents in elastic have an embedding associated.

The current error message is quite cryptic. Let's catch it and make it more descriptive so that the user knows why the query failed. 

**Error message**
Error RequestError(400, 'search_phase_execution_exception', 'runtime error')
**Expected behavior**
A clear and concise description of what you expected to happen.

**Additional context**
Add any other context about the problem here, like document types / preprocessing steps / settings of reader etc.

**To Reproduce**
Use a DPR retriever on an ElasticsearchDocumentStore with docs that don't have an embedding stored yet. (e.g. our docker image `deepset/elasticsearch-game-of-thrones`) 

**System:**
 - OS: -
 - GPU/CPU: -
 - Haystack version (commit or version number): latest master
 - DocumentStore: elastic
 - Reader: -
 - Retriever: DPR or EmbeddingRetriever



Discussion:
Can you please look into the new comment . #1167

Resolving patch:
diff --git a/haystack/document_store/elasticsearch.py b/haystack/document_store/elasticsearch.py
--- a/haystack/document_store/elasticsearch.py
+++ b/haystack/document_store/elasticsearch.py
@@ -4,6 +4,7 @@
 from copy import deepcopy
 from string import Template
 from typing import List, Optional, Union, Dict, Any, Generator
+
 from elasticsearch import Elasticsearch, RequestsHttpConnection
 from elasticsearch.helpers import bulk, scan
 from elasticsearch.exceptions import RequestError
@@ -805,7 +806,15 @@ def query_by_embedding(self,
                 body["_source"] = {"excludes": excluded_meta_data}
 
             logger.debug(f"Retriever query: {body}")
-            result = self.client.search(index=index, body=body, request_timeout=300)["hits"]["hits"]
+            try:
+                result = self.client.search(index=index, body=body, request_timeout=300)["hits"]["hits"]
+            except RequestError as e:
+                if e.error == "search_phase_execution_exception":
+                    error_message: str = "search_phase_execution_exception: Likely some of your stored documents don't have embeddings." \
+                                         " Run the document store's update_embeddings() method."
+                    raise RequestError(e.status_code, error_message, e.info)
+                else:
+                    raise e
 
             documents = [
                 self._convert_es_hit_to_document(hit, adapt_score_for_embedding=True, return_embedding=return_embedding)


Test patch:
Repository: deepset-ai/haystack

Issue:
DPR training fails on multiple GPU's
**Describe the bug**
Following the DPR training [guide](https://haystack.deepset.ai/tutorials/train-dpr), in case trying to run on multiple GPU's, training will fail with a "AttributeError". Trying to call a `BiAdaptiveModel` method on a `DataParallel` object instance. 

**Error message**
AttributeError: 'DataParallel' object has no attribute 'connect_heads_with_processor'

**Expected behavior**
No error

**System:**
 - OS: Ubuntu
 - GPU/CPU: TITAN Xp
 - Haystack version (commit or version number): v0.10.0
 - DocumentStore: InMemoryDocumentStore
 - Reader: 'NA'
 - Retriever: DensePassageRetriever



Discussion:

Resolving patch:
diff --git a/haystack/nodes/retriever/dense.py b/haystack/nodes/retriever/dense.py
--- a/haystack/nodes/retriever/dense.py
+++ b/haystack/nodes/retriever/dense.py
@@ -367,7 +367,10 @@ def train(self,
         self.processor.num_hard_negatives = num_hard_negatives
         self.processor.num_positives = num_positives
 
-        self.model.connect_heads_with_processor(self.processor.tasks, require_labels=True)
+        if isinstance(self.model, DataParallel):
+            self.model.module.connect_heads_with_processor(self.processor.tasks, require_labels=True)
+        else:
+            self.model.connect_heads_with_processor(self.processor.tasks, require_labels=True)
 
         data_silo = DataSilo(processor=self.processor, batch_size=batch_size, distributed=False, max_processes=max_processes)
 
@@ -820,7 +823,10 @@ def train(self,
         self.processor.num_hard_negatives = num_hard_negatives
         self.processor.num_positives = num_positives
 
-        self.model.connect_heads_with_processor(self.processor.tasks, require_labels=True)
+        if isinstance(self.model, DataParallel):
+            self.model.module.connect_heads_with_processor(self.processor.tasks, require_labels=True)
+        else:
+            self.model.connect_heads_with_processor(self.processor.tasks, require_labels=True)
 
         data_silo = DataSilo(processor=self.processor, batch_size=batch_size, distributed=False,
                              max_processes=max_processes)


Test patch: