Repository: deepset-ai/haystack

Issue:
TransformersTranslator is exaggerating its responsibilities and changing Inputs
### Problem:

- When you use the **TransformersTranslator** node, and send a List of Documents, after translation the node **correctly output the translated texts**. But, it's also **changing the input**. So, we got `inputs == outputs` after using the node.

Example code:

```python
DOCS_PT = [
    Document(content="A reiteração que aqui é exigida é a de que, em diversas ocasiões, o Supremo Tribunal venha decidindo uma matéria com maioria e, em determinado momento, por provocação ou de ofício, resolva editar a súmula, buscando o qualificado quorum de dois terços."),
    Document(content="Veda-se, desse modo, a possibilidade da edição de uma súmula vinculante com fundamento em decisão judicial isolada, pois  necessário que ela reflita uma jurisprudência do Tribunal, ou seja, reiterados julgados no mesmo sentido, é dizer, com a mesma interpretação."),
    Document(content="Por mais relevante que seja a matéria, não se concebe a edição de um enunciado sumular vinculante no primeiro ou no segundo julgamento."),
    Document(content="Trata-se de uma questão democrática, que tem na divisão de Poderes o seu ponto central."),
    Document(content="Pudesse o STF emitir súmulas vinculantes a qualquer momento, estar-se-ia admitindo que a discussão pelo restante do sistema teria sido tornada despicienda."),
    Document(content="O Supremo Tribunal, à revelia da Constituição, estaria estabelecendo um adiantamento liminar de sentido.")
]

pt_to_en = TransformersTranslator(model_name_or_path="Helsinki-NLP/opus-mt-mul-en", use_gpu=True)
DOCS_EN = pt_to_en.translate(documents=DOCS_PT, query=None)
```

### Past discussion:

@ZanSara 

> The original aim, most likely, was to make the document store contain only documents in a uniform language, to allow the Retriever to work properly. In fact, if the retriever is monolingual, documents written in other languages can break it. So Translator is making sure that the original document, written in another language, is no more present, to avoid this scenario.
> On the other hand though, I don't think this is the right way to handle the situation :sorrindo_suor: I can see two ways we can improve this:
> Move or duplicate the original text into the metadata (so at least it can be retrieved later) - this could be a quick and easy fix for your situation. It could even be done without modifying Haystack, for now.
> Check if one can filter documents by language before sending them to the Retriever. This could be implemented in the Retriever themselves, or by a new node, depending on how is it done.

@danielbichuetti 

> Inputs and Outputs are two different concepts on haystack, specially how it's coded. There is not any other pipe which returns outputs and at same time replaces inputs.
> Indeed, the other method behavior, query, doesn't replace the query itself.
> Let's think into 51 million American court documents which were indexed. User then decides to translate these documents to German, so he can start to work on a new dataset to get one distilled model in German.  User is currently making usage of any document store. After he runs the pipeline, he just discovers he lost all the original documents with Transformers translation. And translating it back, he gets just a piece of the original.
> Maybe we should let any issues regarding how the user stores documents to the document store itself.

```python
es_doc_store_pt = ElasticsearchDocumentStore(host="some.es.cloud.domain.ai", scheme="https", username="luigi", password="mario", index="court_decisions_pt", analyzer="pt")

DOCS_PT = [
    Document(content="A reiteração que aqui é exigida é a de que, em diversas ocasiões, o Supremo Tribunal venha decidindo uma matéria com maioria e, em determinado momento, por provocação ou de ofício, resolva editar a súmula, buscando o qualificado quorum de dois terços."),
    Document(content="Veda-se, desse modo, a possibilidade da edição de uma súmula vinculante com fundamento em decisão judicial isolada, pois  necessário que ela reflita uma jurisprudência do Tribunal, ou seja, reiterados julgados no mesmo sentido, é dizer, com a mesma interpretação."),
    Document(content="Por mais relevante que seja a matéria, não se concebe a edição de um enunciado sumular vinculante no primeiro ou no segundo julgamento."),
    Document(content="Trata-se de uma questão democrática, que tem na divisão de Poderes o seu ponto central."),
    Document(content="Pudesse o STF emitir súmulas vinculantes a qualquer momento, estar-se-ia admitindo que a discussão pelo restante do sistema teria sido tornada despicienda."),
    Document(content="O Supremo Tribunal, à revelia da Constituição, estaria estabelecendo um "adiantamento liminar de sentido.")
]

pt_to_en = TransformersTranslator(model_name_or_path="Helsinki-NLP/opus-mt-mul-en", use_gpu=True)
res_helsinki_pt_en = pt_to_en.translate(documents=DOCS_PT, query=None)


es_doc_store_en = ElasticsearchDocumentStore(host="some.es.cloud.domain.ai", scheme="https", username="luigi", password="mario", index="court_decisions_en", analyzer="en")

ec_doc_store_en.write_documents(res_helsinki_pt_en)
```

> This is just a sample, but a point about how easy it would be to let the responsibility to the developer.

Here we get to some important considerations about the misusage of the translation mechanisms and why that behavior was probably first implemented:

@ZanSara 

> However, to stay in your example, having the original English documents given to a German retriever would result in garbage output. That's why the Translator was implemented this way: to prevent people from making such mistake. I don't like this, I also think this is responsibility of another node


### Proposed solution draft:

- The main word for this change is **decoupling**. It's not a good software engineering decision, generally, to allow a method to output something and at the same time change the input to the same data, going backward on the chain. Specially when there is no documentation, a strong and unique rare case where there is no other possibility.

As per @ZanSara considerations, there are problems if user translates and misuse this results, writing to the same Document Store without any way to differentiate. There could happen to show up cases with a bunch of multi-language into same index of Elasticsearch (for example). But I would like to make a note that this could only happen if the user chooses to do the bad act, not by haystack doing quirks on the chain of responsibility.

My proposal for it:

- Get Translator node to work as all other nodes, and even as itself when related to query and not document. Output translated documents and keep inputs like they were sent.
- Make a proper documentation on the Translator node page about the issues of mixing in the same document store without some filtering mechanisms to differentiate languages when using Retriever. 
- Include one example of using RouteDocuments to send to different retrievers based on some metadata (language in this case).


Discussion:
So, may I move forward and start this change? @sjrl 
Yes that sounds good to me!

Resolving patch:
diff --git a/haystack/nodes/translator/transformers.py b/haystack/nodes/translator/transformers.py
--- a/haystack/nodes/translator/transformers.py
+++ b/haystack/nodes/translator/transformers.py
@@ -1,8 +1,9 @@
 import logging
+from copy import deepcopy
 from typing import Any, Dict, List, Optional, Union
 
 from tqdm.auto import tqdm
-from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
+from transformers import AutoModelForSeq2SeqLM, AutoTokenizer  # type: ignore
 
 from haystack.errors import HaystackError
 from haystack.schema import Document, Answer
@@ -77,7 +78,7 @@ def __init__(
 
     def translate(
         self,
-        results: List[Dict[str, Any]] = None,
+        results: Optional[List[Dict[str, Any]]] = None,
         query: Optional[str] = None,
         documents: Optional[Union[List[Document], List[Answer], List[str], List[Dict[str, Any]]]] = None,
         dict_key: Optional[str] = None,
@@ -95,10 +96,10 @@ def translate(
             queries_for_translator = [result["query"] for result in results]
             answers_for_translator = [result["answers"][0].answer for result in results]
         if not query and not documents and results is None:
-            raise AttributeError("Translator needs query or documents to perform translation.")
+            raise AttributeError("Translator needs a query or documents to perform translation.")
 
         if query and documents:
-            raise AttributeError("Translator needs either query or documents but not both.")
+            raise AttributeError("Translator needs either a query or documents but not both.")
 
         if documents and len(documents) == 0:
             logger.warning("Empty documents list is passed")
@@ -144,17 +145,22 @@ def translate(
             if isinstance(documents, list) and isinstance(documents[0], str):
                 return [translated_text for translated_text in translated_texts]
 
+            translated_documents: Union[
+                List[Document], List[Answer], List[str], List[Dict[str, Any]]
+            ] = []  # type: ignore
             for translated_text, doc in zip(translated_texts, documents):
-                if isinstance(doc, Document):
-                    doc.content = translated_text
-                elif isinstance(doc, Answer):
-                    doc.answer = translated_text
+                translated_document = deepcopy(doc)
+                if isinstance(translated_document, Document):
+                    translated_document.content = translated_text
+                elif isinstance(translated_document, Answer):
+                    translated_document.answer = translated_text
                 else:
-                    doc[dict_key] = translated_text  # type: ignore
+                    translated_document[dict_key] = translated_text  # type: ignore
+                translated_documents.append(translated_document)  # type: ignore
 
-            return documents
+            return translated_documents
 
-        raise AttributeError("Translator need query or documents to perform translation")
+        raise AttributeError("Translator needs a query or documents to perform translation")
 
     def translate_batch(
         self,
@@ -175,11 +181,11 @@ def translate_batch(
             raise AttributeError("Translator needs either query or documents but not both.")
 
         if not queries and not documents:
-            raise AttributeError("Translator needs query or documents to perform translation.")
+            raise AttributeError("Translator needs a query or documents to perform translation.")
 
+        translated = []
         # Translate queries
         if queries:
-            translated = []
             for query in tqdm(queries, disable=not self.progress_bar, desc="Translating"):
                 cur_translation = self.translate(query=query)
                 translated.append(cur_translation)
@@ -188,14 +194,13 @@ def translate_batch(
         elif documents:
             # Single list of documents / answers
             if not isinstance(documents[0], list):
-                translated = self.translate(documents=documents)  # type: ignore
+                translated.append(self.translate(documents=documents))  # type: ignore
             # Multiple lists of document / answer lists
             else:
-                translated = []
                 for cur_list in tqdm(documents, disable=not self.progress_bar, desc="Translating"):
                     if not isinstance(cur_list, list):
                         raise HaystackError(
-                            f"cur_list was of type {type(cur_list)}, but expected a list of " f"Documents / Answers."
+                            f"cur_list was of type {type(cur_list)}, but expected a list of Documents / Answers."
                         )
                     cur_translation = self.translate(documents=cur_list)
                     translated.append(cur_translation)


Test patch:
diff --git a/test/nodes/test_translator.py b/test/nodes/test_translator.py
--- a/test/nodes/test_translator.py
+++ b/test/nodes/test_translator.py
@@ -5,6 +5,8 @@
 EXPECTED_OUTPUT = "Ich lebe in Berlin"
 INPUT = "I live in Berlin"
 
+DOCUMENT_INPUT = Document(content=INPUT)
+
 
 def test_translator_with_query(en_to_de_translator):
     assert en_to_de_translator.translate(query=INPUT) == EXPECTED_OUTPUT
@@ -18,10 +20,22 @@ def test_translator_with_document(en_to_de_translator):
     assert en_to_de_translator.translate(documents=[Document(content=INPUT)])[0].content == EXPECTED_OUTPUT
 
 
+def test_translator_with_document_preserves_input(en_to_de_translator):
+    original_document = Document(content=INPUT)
+    en_to_de_translator.translate(documents=[original_document])[0]  # pylint: disable=expression-not-assigned
+    assert original_document.content == INPUT
+
+
 def test_translator_with_dictionary(en_to_de_translator):
     assert en_to_de_translator.translate(documents=[{"content": INPUT}])[0]["content"] == EXPECTED_OUTPUT
 
 
+def test_translator_with_dictionary_preserves_input(en_to_de_translator):
+    original_document = {"content": INPUT}
+    en_to_de_translator.translate(documents=[original_document])[0]  # pylint: disable=expression-not-assigned
+    assert original_document["content"] == INPUT
+
+
 def test_translator_with_dictionary_with_dict_key(en_to_de_translator):
     assert en_to_de_translator.translate(documents=[{"key": INPUT}], dict_key="key")[0]["key"] == EXPECTED_OUTPUT