Repository: deepset-ai/haystack

Issue:
Preprocessing should allow keeping Document ids unchanged
Currently, preprocessing components such as the DocumentCleaner get documents as input and return preprocessed documents **but with different ids**.
We should enable users to specify custom ids and make sure these ids are used when writing the documents to a database.
We could achieve that in different ways:

1. Ensure that preprocessing components such as the DocumentCleaner keep the document ids unchanged. However that means the id is not based on the preprocessed content anymore but on the original content.
2. Add a new Component that calculates ids of documents right before the DocumentWriter. That means users with custom ids could provide a list of ids as inputs


A current workaround is to define a custom component that includes a method returning custom ids:

```python
from copy import deepcopy
from typing import List
from haystack import Document, component

@component
class DocumentIDCreator:
    # This is a pass-through component that returns the same Documents but with different ids
    @component.output_types(documents=List[Document])
    def run(self, documents: List[Document]):
        updated_documents = []
        for doc in documents:
            id = calculate_id(doc, id_hash_keys) # your code
            updated_documents.append(Document(id=id, content=doc.content, meta=deepcopy(doc.meta), embedding=doc.embedding))

        return {"documents": updated_documents}
```

Discussion:
coveralls: ## Pull Request Test Coverage Report for [Build 9109726478](https://coveralls.io/builds/67525794)


### Details

* **0** of **0**   changed or added relevant lines in **0** files are covered.
* **1** unchanged line in **1** file lost coverage.
* Overall coverage increased (+**0.002%**) to **90.587%**

---


|  Files with Coverage Reduction | New Missed Lines | % |
| :-----|--------------|--: |
| [components/preprocessors/document_cleaner.py](https://coveralls.io/builds/67525794/source?filename=components%2Fpreprocessors%2Fdocument_cleaner.py#L250) | 1 | 98.82% |
<!-- | **Total:** | **1** |  | -->

|  Totals | [![Coverage Status](https://coveralls.io/builds/67525794/badge)](https://coveralls.io/builds/67525794) |
| :-- | --: |
| Change from base [Build 9103206687](https://coveralls.io/builds/67516278): |  0.002% |
| Covered Lines: | 6592 |
| Relevant Lines: | 7277 |

---
##### 💛  - [Coveralls](https://coveralls.io)

vblagoje: Thank you @CarlosFerLo I'll let @silvanocerza approve this one and remove myself from PR review. Thanks for your contribution 🙏 

Patch:
diff --git a/haystack/components/preprocessors/document_cleaner.py b/haystack/components/preprocessors/document_cleaner.py
index 80f281be..4248862d 100644
--- a/haystack/components/preprocessors/document_cleaner.py
+++ b/haystack/components/preprocessors/document_cleaner.py
@@ -40,6 +40,7 @@ class DocumentCleaner:
         remove_empty_lines: bool = True,
         remove_extra_whitespaces: bool = True,
         remove_repeated_substrings: bool = False,
+        keep_id: bool = False,
         remove_substrings: Optional[List[str]] = None,
         remove_regex: Optional[str] = None,
     ):
@@ -53,6 +54,7 @@ class DocumentCleaner:
             which is supported by `TextFileToDocument` and `AzureOCRDocumentConverter`.
         :param remove_substrings: List of substrings to remove from the text.
         :param remove_regex: Regex to match and replace substrings by "".
+        :param keep_id: keep the ids of the original documents
         """
 
         self.remove_empty_lines = remove_empty_lines
@@ -60,6 +62,7 @@ class DocumentCleaner:
         self.remove_repeated_substrings = remove_repeated_substrings
         self.remove_substrings = remove_substrings
         self.remove_regex = remove_regex
+        self.keep_id = keep_id
 
     @component.output_types(documents=List[Document])
     def run(self, documents: List[Document]):
@@ -98,7 +101,7 @@ class DocumentCleaner:
             if self.remove_repeated_substrings:
                 text = self._remove_repeated_substrings(text)
 
-            cleaned_docs.append(Document(content=text, meta=deepcopy(doc.meta)))
+            cleaned_docs.append(Document(content=text, meta=deepcopy(doc.meta), id=doc.id if self.keep_id else ""))
 
         return {"documents": cleaned_docs}
 
diff --git a/releasenotes/notes/add-keep-id-to-document-cleaner-2a9854b5f195bb78.yaml b/releasenotes/notes/add-keep-id-to-document-cleaner-2a9854b5f195bb78.yaml
new file mode 100644
index 00000000..805c811d
--- /dev/null
+++ b/releasenotes/notes/add-keep-id-to-document-cleaner-2a9854b5f195bb78.yaml
@@ -0,0 +1,4 @@
+---
+enhancements:
+  - |
+    The `DocumentCleaner` class has the optional attribute `keep_id` that if set to True it keeps the document ids unchanged after cleanup.


Test patch:
diff --git a/test/components/preprocessors/test_document_cleaner.py b/test/components/preprocessors/test_document_cleaner.py
index 2aaf2f37..0acd9e8e 100644
--- a/test/components/preprocessors/test_document_cleaner.py
+++ b/test/components/preprocessors/test_document_cleaner.py
@@ -17,6 +17,7 @@ class TestDocumentCleaner:
         assert cleaner.remove_repeated_substrings is False
         assert cleaner.remove_substrings is None
         assert cleaner.remove_regex is None
+        assert cleaner.keep_id is False
 
     def test_non_text_document(self, caplog):
         with caplog.at_level(logging.WARNING):
@@ -130,3 +131,11 @@ class TestDocumentCleaner:
         for doc, cleaned_doc in zip(documents, result["documents"]):
             assert doc.meta == cleaned_doc.meta
             assert cleaned_doc.content == "Text."
+
+    def test_keep_id_does_not_alter_document_ids(self):
+        cleaner = DocumentCleaner(keep_id=True)
+        documents = [Document(content="Text. ", id="1"), Document(content="Text. ", id="2")]
+        result = cleaner.run(documents=documents)
+        assert len(result["documents"]) == 2
+        assert result["documents"][0].id == "1"
+        assert result["documents"][1].id == "2"