Repository: deepset-ai/haystack

Issue:
Elasticsearch compatibility: FAQ-style QA search_phase_execution_exception, script_score script returned an invalid score
**Describe the bug**
When I attempt to call get_answers_via_similar_questions using FAQ-style QA an error is thrown. This occurs when I use two separate indices, one for QA-style and another for FAQ-style.  I want to leverage a QA instance and an FAQ with their own separate indices for experimental purposes.

**Error message**
```
10/13/2020 06:43:06 - WARNING - elasticsearch -   POST http://localhost:9200/faq/_search [status:400 request:0.130s]
Traceback (most recent call last):
  File "chatbots/haystack/faqbot.py", line 214, in <module>
    main()
  File "chatbots/haystack/faqbot.py", line 186, in main
    faqbot.interact(question)
  File "chatbots/haystack/faqbot.py", line 57, in interact
    prediction = finder.get_answers_via_similar_questions(question=question, top_k_retriever=1)
  File "/Users/aaronbriel/chatbots/haystack/.venv/src/farm-haystack/haystack/finder.py", line 95, in get_answers_via_similar_questions
    documents = self.retriever.retrieve(question, top_k=top_k_retriever, filters=filters, index=index)
  File "/Users/aaronbriel/chatbots/haystack/.venv/src/farm-haystack/haystack/retriever/dense.py", line 327, in retrieve
    top_k=top_k, index=index)
  File "/Users/aaronbriel/chatbots/haystack/.venv/src/farm-haystack/haystack/document_store/elasticsearch.py", line 465, in query_by_embedding
    result = self.client.search(index=index, body=body, request_timeout=300)["hits"]["hits"]
  File "/Users/aaronbriel/chatbots/haystack/.venv/lib/python3.7/site-packages/elasticsearch/client/utils.py", line 152, in _wrapped
    return func(*args, params=params, headers=headers, **kwargs)
  File "/Users/aaronbriel/chatbots/haystack/.venv/lib/python3.7/site-packages/elasticsearch/client/__init__.py", line 1617, in search
    body=body,
  File "/Users/aaronbriel/chatbots/haystack/.venv/lib/python3.7/site-packages/elasticsearch/transport.py", line 392, in perform_request
    raise e
  File "/Users/aaronbriel/chatbots/haystack/.venv/lib/python3.7/site-packages/elasticsearch/transport.py", line 365, in perform_request
    timeout=timeout,
  File "/Users/aaronbriel/chatbots/haystack/.venv/lib/python3.7/site-packages/elasticsearch/connection/http_urllib3.py", line 269, in perform_request
    self._raise_error(response.status, raw_data)
  File "/Users/aaronbriel/chatbots/haystack/.venv/lib/python3.7/site-packages/elasticsearch/connection/base.py", line 301, in _raise_error
    status_code, error_message, additional_info
elasticsearch.exceptions.RequestError: RequestError(400, 'search_phase_execution_exception', 'script_score script returned an invalid score [-3.4487913] for doc [41]. Must be a non-negative score!')
make: *** [ask_faq] Error 1
```

**Expected behavior**
I expected the request to return results.

**Additional context**
I'm using code very similar to Tutorial4_FAQ_style_QA.ipynb and  Tutorial1_Basic_QA_Pipeline.ipynb, except for one difference. I first store data in elasticsearch using 2 indices (using the same storage code for each tutorial above), once for the QA-style and then for the FAQ-style. Each has their own unique elasticsearch index.

When I run a query to the QA get_answers method the expected result is returned. However, when I run the query to the FAQ get_answers_via_similar_questions I see the error. Looking at #408 I checked the mapping of the FAQ index which seems correct:
```
{
   "faq":{
      "mappings":{
         "dynamic_templates":[
            {
               "strings":{
                  "path_match":"*",
                  "match_mapping_type":"string",
                  "mapping":{
                     "type":"keyword"
                  }
               }
            }
         ],
         "properties":{
            "answer_html":{
               "type":"keyword"
            },
            "category":{
               "type":"keyword"
            },
            "city":{
               "type":"keyword"
            },
            "country":{
               "type":"keyword"
            },
            "lang":{
               "type":"keyword"
            },
            "last_update":{
               "type":"keyword"
            },
            "link":{
               "type":"keyword"
            },
            "name":{
               "type":"keyword"
            },
            "question":{
               "type":"keyword"
            },
            "question_emb":{
               "type":"dense_vector",
               "dims":768
            },
            "region":{
               "type":"keyword"
            },
            "source":{
               "type":"keyword"
            },
            "text":{
               "type":"text"
            }
         }
      }
   }
}
```

**To Reproduce**
1. Instantiate QA ElasticsearchDocumentStore, ElasticsearchRetriever, FarmReader and store SQUAD style json using convert_files_to_dicts and document_store.write_documents(dicts) following Tutorial1_Basic_QA_Pipeline.ipynb but with custom index
2. Instantiate FAQ ElasticsearchDocumentStore and EmbeddingRetriever and store csv (faq_covidbert.csv) using convert_files_to_dicts and document_store.write_documents(dicts) following Tutorial4_FAQ_style_QA.ipynb but with custom index
3. Call get_answers_via_similar_questions with top_k_retriever=1

**System:**
 - OS: 
 - GPU/CPU:
 - Haystack version (commit or version number): latest master branch
 - DocumentStore: ElasticsearchDocumentStore
 - Reader: None
 - Retriever: EmbeddingRetriever



Discussion:
I've discovered that the issue is due to the version of elasticsearch. In my docker-compose I was referencing version 7.9.2. When I reverted this to 7.6.2 the issue was resolved. So it looks like `get_answers` is compatible with 7.9.2 but `get_answers_via_similar_questions` is not.
I will close this - I'm guessing you are aware of the compatibility issue...
@tholor @tanaysoni better to address this compatibility issue
Yes, let's fix it

Resolving patch:
diff --git a/haystack/document_store/elasticsearch.py b/haystack/document_store/elasticsearch.py
--- a/haystack/document_store/elasticsearch.py
+++ b/haystack/document_store/elasticsearch.py
@@ -460,7 +460,8 @@ def query_by_embedding(self,
                     "script_score": {
                         "query": {"match_all": {}},
                         "script": {
-                            "source": f"{self.similarity_fn_name}(params.query_vector,'{self.embedding_field}') + 1.0",
+                            # offset score to ensure a positive range as required by Elasticsearch
+                            "source": f"{self.similarity_fn_name}(params.query_vector,'{self.embedding_field}') + 1000",
                             "params": {
                                 "query_vector": query_emb.tolist()
                             }
@@ -497,7 +498,7 @@ def _convert_es_hit_to_document(self, hit: dict, adapt_score_for_embedding: bool
         score = hit["_score"] if hit["_score"] else None
         if score:
             if adapt_score_for_embedding:
-                score -= 1
+                score -= 1000
                 probability = (score + 1) / 2  # scaling probability from cosine similarity
             else:
                 probability = float(expit(np.asarray(score / 8)))  # scaling probability from TFIDF/BM25
diff --git a/tutorials/Tutorial4_FAQ_style_QA.py b/tutorials/Tutorial4_FAQ_style_QA.py
--- a/tutorials/Tutorial4_FAQ_style_QA.py
+++ b/tutorials/Tutorial4_FAQ_style_QA.py
@@ -26,7 +26,7 @@
 if LAUNCH_ELASTICSEARCH:
     logging.info("Starting Elasticsearch ...")
     status = subprocess.run(
-        ['docker run -d -p 9200:9200 -e "discovery.type=single-node" elasticsearch:7.6.2'], shell=True
+        ['docker run -d -p 9200:9200 -e "discovery.type=single-node" elasticsearch:7.9.2'], shell=True
     )
     if status.returncode:
         raise Exception("Failed to launch Elasticsearch. If you want to connect to an existing Elasticsearch instance"
diff --git a/tutorials/Tutorial5_Evaluation.py b/tutorials/Tutorial5_Evaluation.py
--- a/tutorials/Tutorial5_Evaluation.py
+++ b/tutorials/Tutorial5_Evaluation.py
@@ -34,7 +34,7 @@
 if LAUNCH_ELASTICSEARCH:
     logging.info("Starting Elasticsearch ...")
     status = subprocess.run(
-        ['docker run -d -p 9200:9200 -e "discovery.type=single-node" elasticsearch:7.6.2'], shell=True
+        ['docker run -d -p 9200:9200 -e "discovery.type=single-node" elasticsearch:7.9.2'], shell=True
     )
     if status.returncode:
         raise Exception("Failed to launch Elasticsearch. If you want to connect to an existing Elasticsearch instance"


Test patch:
diff --git a/test/conftest.py b/test/conftest.py
--- a/test/conftest.py
+++ b/test/conftest.py
@@ -33,7 +33,7 @@ def elasticsearch_fixture():
             shell=True
         )
         status = subprocess.run(
-            ['docker run -d --name haystack_test_elastic -p 9200:9200 -e "discovery.type=single-node" elasticsearch:7.9.1'],
+            ['docker run -d --name haystack_test_elastic -p 9200:9200 -e "discovery.type=single-node" elasticsearch:7.9.2'],
             shell=True
         )
         if status.returncode:
@@ -160,12 +160,12 @@ def document_store(request, test_docs_xs, elasticsearch_fixture):
     return get_document_store(request.param)
 
 
-@pytest.fixture(params=["es_filter_only", "elsticsearch", "dpr", "embedding", "tfidf"])
+@pytest.fixture(params=["es_filter_only", "elasticsearch", "dpr", "embedding", "tfidf"])
 def retriever(request, document_store):
     return get_retriever(request.param, document_store)
 
 
-@pytest.fixture(params=["es_filter_only", "elsticsearch", "dpr", "embedding", "tfidf"])
+@pytest.fixture(params=["es_filter_only", "elasticsearch", "dpr", "embedding", "tfidf"])
 def retriever_with_docs(request, document_store_with_docs):
     return get_retriever(request.param, document_store_with_docs)
 
@@ -206,7 +206,7 @@ def get_retriever(retriever_type, document_store):
         retriever = EmbeddingRetriever(document_store=document_store,
                                        embedding_model="deepset/sentence_bert",
                                        use_gpu=False)
-    elif retriever_type == "elsticsearch":
+    elif retriever_type == "elasticsearch":
         retriever = ElasticsearchRetriever(document_store=document_store)
     elif retriever_type == "es_filter_only":
         retriever = ElasticsearchFilterOnlyRetriever(document_store=document_store)
diff --git a/test/test_elastic_retriever.py b/test/test_elastic_retriever.py
--- a/test/test_elastic_retriever.py
+++ b/test/test_elastic_retriever.py
@@ -2,7 +2,7 @@
 
 
 @pytest.mark.parametrize("document_store_with_docs", [("elasticsearch")], indirect=True)
-@pytest.mark.parametrize("retriever_with_docs", ["elsticsearch"], indirect=True)
+@pytest.mark.parametrize("retriever_with_docs", ["elasticsearch"], indirect=True)
 def test_elasticsearch_retrieval(retriever_with_docs, document_store_with_docs):
     res = retriever_with_docs.retrieve(query="Who lives in Berlin?")
     assert res[0].text == "My name is Carla and I live in Berlin"
@@ -11,7 +11,7 @@ def test_elasticsearch_retrieval(retriever_with_docs, document_store_with_docs):
 
 
 @pytest.mark.parametrize("document_store_with_docs", [("elasticsearch")], indirect=True)
-@pytest.mark.parametrize("retriever_with_docs", ["elsticsearch"], indirect=True)
+@pytest.mark.parametrize("retriever_with_docs", ["elasticsearch"], indirect=True)
 def test_elasticsearch_retrieval_filters(retriever_with_docs, document_store_with_docs):
     res = retriever_with_docs.retrieve(query="Who lives in Berlin?", filters={"name": ["filename1"]})
     assert res[0].text == "My name is Carla and I live in Berlin"
diff --git a/test/test_eval.py b/test/test_eval.py
--- a/test/test_eval.py
+++ b/test/test_eval.py
@@ -61,7 +61,7 @@ def test_eval_reader(reader, document_store: BaseDocumentStore):
 
 @pytest.mark.parametrize("document_store", ["elasticsearch"], indirect=True)
 @pytest.mark.parametrize("open_domain", [True, False])
-@pytest.mark.parametrize("retriever", ["elsticsearch"], indirect=True)
+@pytest.mark.parametrize("retriever", ["elasticsearch"], indirect=True)
 def test_eval_elastic_retriever(document_store: BaseDocumentStore, open_domain, retriever):
     # add eval data (SQUAD format)
     document_store.delete_all_documents(index="test_eval_document")
@@ -81,7 +81,7 @@ def test_eval_elastic_retriever(document_store: BaseDocumentStore, open_domain,
 
 @pytest.mark.parametrize("document_store", ["elasticsearch"], indirect=True)
 @pytest.mark.parametrize("reader", ["farm"], indirect=True)
-@pytest.mark.parametrize("retriever", ["elsticsearch"], indirect=True)
+@pytest.mark.parametrize("retriever", ["elasticsearch"], indirect=True)
 def test_eval_finder(document_store: BaseDocumentStore, reader, retriever):
     finder = Finder(reader=reader, retriever=retriever)