Repository: deepset-ai/haystack

Issue:
`EmbeddingRetriever` assumes the title of the document is in a field of `meta` called `name`
**Describe the bug**
- In `embed_documents`, `EmbeddingRetriever` assumes the title of the document is in a field of `meta` called `name`: https://github.com/deepset-ai/haystack/blob/7e79a4854085fc8fd5875266ec4ce466b8313197/haystack/nodes/retriever/_embedding_encoder.py#L189

**Expected behavior**
- The name of the field containing the title should be configurable.
- We should be able to configure which meta fields are embedded through the `embed_meta_fields` parameter.

**Additional context**
- https://github.com/deepset-ai/haystack/pull/3252#discussion_r976248153




Discussion:
Hey @ZanSara!

I think that this line is **superfluous** and indeed is probably wrong:
https://github.com/deepset-ai/haystack/blob/6cb4e939653719698764b2e0ad379b0859f862be/haystack/nodes/retriever/_embedding_encoder.py#L214


In fact, before calling that method, in the `embed_documents` method (of the `EmbeddingRetriever`), `_preprocess_documents` is called and the meta fields indicated by `embed_meta_fields` are **concatenated there**.
https://github.com/deepset-ai/haystack/blob/6cb4e939653719698764b2e0ad379b0859f862be/haystack/nodes/retriever/dense.py#L1838-L1868

So for now, if you have such a document:
```json
{"content": "mycontent",
"meta": {"name": "myname"}}
```
and you try to embed it with an `EmbeddingRetriever`, by specifying `embed_meta_fields=['name']`,
under the hood these are the passages to encode (sum of `_preprocess_documents` and `embed_documents`).
```python
[['myname', 'myname\ncontent']]
```
I verified that with some experiments.

### Possible solution
We can simply remove that line from `embed_documents` if you agree that the meta fields have already been concatenated in `_preprocess_documents`.
Do we want to provide some default value for `EmbeddingRetriever`'s `embed_meta_fields` attribute? Currently, it is an empty list.
Hey @anakin87, wow, thank you for digging into this one.

So, although I don't believe this duplication benefits anyone, there might be parts of the code deep into the retriever that either assume the name will be in the first position of the list, and others that assume the name is already concatenated. My recommendation would be to proceed, remove the line, and then thoroughly test the resulting code. By this, I mean that we should make sure all tutorials run on it. If you can, running all tests locally would be also fantastic.

 Do you think you can do that? If it's not viable I'll do the tests myself, let me know. It will just take a bit longer as I have a lot to catch up after my holidays :smile: 
Mayank is already working on this.
@mayankjobanputra
@julian-risch thanks! I wasn't aware yet :+1: 

Resolving patch:
diff --git a/haystack/nodes/retriever/_embedding_encoder.py b/haystack/nodes/retriever/_embedding_encoder.py
--- a/haystack/nodes/retriever/_embedding_encoder.py
+++ b/haystack/nodes/retriever/_embedding_encoder.py
@@ -187,8 +187,8 @@ def __init__(self, retriever: "EmbeddingRetriever"):
                 f"This can be set when initializing the DocumentStore"
             )
 
-    def embed(self, texts: Union[List[List[str]], List[str], str]) -> np.ndarray:
-        # texts can be a list of strings or a list of [title, text]
+    def embed(self, texts: Union[List[str], str]) -> np.ndarray:
+        # texts can be a list of strings
         # get back list of numpy embedding vectors
         emb = self.embedding_model.encode(
             texts, batch_size=self.batch_size, show_progress_bar=self.show_progress_bar, convert_to_numpy=True
@@ -211,7 +211,7 @@ def embed_documents(self, docs: List[Document]) -> np.ndarray:
         :param docs: List of documents to embed.
         :return: Embeddings, one per input document, shape: (documents, embedding_dim)
         """
-        passages = [[d.meta["name"] if d.meta and "name" in d.meta else "", d.content] for d in docs]
+        passages = [d.content for d in docs]
         return self.embed(passages)
 
     def train(


Test patch:
diff --git a/test/document_stores/test_document_store.py b/test/document_stores/test_document_store.py
--- a/test/document_stores/test_document_store.py
+++ b/test/document_stores/test_document_store.py
@@ -1448,8 +1448,15 @@ def test_similarity_score_sentence_transformers(document_store_with_docs):
     pipeline = DocumentSearchPipeline(retriever)
     prediction = pipeline.run("Paul lives in New York")
     scores = [document.score for document in prediction["documents"]]
+    assert [document.content for document in prediction["documents"]] == [
+        "My name is Paul and I live in New York",
+        "My name is Matteo and I live in Rome",
+        "My name is Christelle and I live in Paris",
+        "My name is Carla and I live in Berlin",
+        "My name is Camila and I live in Madrid",
+    ]
     assert scores == pytest.approx(
-        [0.8497486114501953, 0.6622999012470245, 0.6077829301357269, 0.5928314849734306, 0.5614184625446796], abs=1e-3
+        [0.9149981737136841, 0.6895168423652649, 0.641706794500351, 0.6206043660640717, 0.5837393924593925], abs=1e-3
     )