Repository: deepset-ai/haystack
Pull request: feat: Add TextDocumentSplitter that splits by word, sentence, passage (2.0)
### Related Issues

- fixes #5675 

### Proposed Changes:

 <!--- In case of a bug: Describe what caused the issue and how you solved it -->
 <!--- In case of a feature: Describe what did you add and how it works -->

### How did you test it?

Added unit tests

### Notes for the reviewer

Behavior is not the same as for the old Preprocessor. For example, cleaning of strings such as removing trailing whitespaces or newlines is not implemented yet in the new TextDocumentSplitter.

This early version is mainly for unblocking @vblagoje and his work on WebRetriever.

Whether nltk is our best option for proper sentence splitting or not is still up for discussion, which is why I went for the naive solution of splitting by ".".

### Checklist

- I have read the [contributors guidelines](https://github.com/deepset-ai/haystack/blob/main/CONTRIBUTING.md) and the [code of conduct](https://github.com/deepset-ai/haystack/blob/main/code_of_conduct.txt)
- I have updated the related issue with new insights and changes
- I added unit tests and updated the docstrings
- I've used one of the [conventional commit types](https://www.conventionalcommits.org/en/v1.0.0/) for my PR title: `fix:`, `feat:`, `build:`, `chore:`, `ci:`, `docs:`, `style:`, `refactor:`, `perf:`, `test:`.
- I documented my code
- I ran [pre-commit hooks](https://github.com/deepset-ai/haystack/blob/main/CONTRIBUTING.md#installation) and fixed any issue


Problem statement:
`TextDocumentSplitter`
Splits the text into chunks long enough that models can process them.

Ideally it should be able to split the text:

- By word, sentence, paragraph
    - Implies sentence tokenization, now done with NLTK
- By token (makes this component tokenizer-aware)
- With customizable overlap
    - Overlap to be measured in different units, like words, tokens, sentences, characters

It should always be possible to put soft and hard limits on the document length, i.e. by max chars or max tokens, regardless of the unit used to split. We should evaluate whether we want such limits to be active by default or not.

`TextDocumentSplitter` should also split the structure dictionary properly if it’s present, so it should retain only the page numbers and headings that are included in each split, by also updating the chars positions.

Draft I/O:

```python
@component
class TextDocumentSplitter:

    @component.output_types(documents=List[Document])
    def run(self, documents: List[Document], ... split_by, split_overlap, etc ...):
        # splits the documents
        return {"documents": documents}
```

Hints from issue discussion:
In addition to the listed requirements it would be great if users could specify a custom string that separates paragraphs in a long input string. For example, paragraphs might not be separated by `\n\n` but by `-----` as mentioned in the following feature request from our community: https://github.com/deepset-ai/haystack/issues/5691

Commit message: add tests

Patch to review:
diff --git a/haystack/preview/components/preprocessors/__init__.py b/haystack/preview/components/preprocessors/__init__.py
new file mode 100644
index 0000000000..33a0e2cd18
--- /dev/null
+++ b/haystack/preview/components/preprocessors/__init__.py
@@ -0,0 +1,3 @@
+from haystack.preview.components.preprocessors.text_document_splitter import TextDocumentSplitter
+
+__all__ = ["TextDocumentSplitter"]
diff --git a/haystack/preview/components/preprocessors/text_document_splitter.py b/haystack/preview/components/preprocessors/text_document_splitter.py
new file mode 100644
index 0000000000..d68adfc7f1
--- /dev/null
+++ b/haystack/preview/components/preprocessors/text_document_splitter.py
@@ -0,0 +1,109 @@
+from typing import List, Optional, Dict, Any, Literal, Tuple
+
+from more_itertools import windowed
+
+from haystack.preview import component, Document, default_from_dict, default_to_dict
+
+
+@component
+class TextDocumentSplitter:
+    """
+    Split a text document into a list of text documents with shorter texts.
+    This is useful for splitting documents with long texts that otherwise would not fit into the maximum text length of language models
+    """
+
+    def __init__(
+        self, split_by: Literal["word", "sentence", "passage"] = "word", split_length: int = 200, split_overlap: int = 0
+    ):
+        """
+        :param split_by: The unit by which the document should be split. Choose from "word", "sentence", "passage".
+        :param split_length: The maximum length of each split.
+        :param split_overlap: The number of units that each split should overlap.
+        """
+        self.split_by = split_by
+        self.split_length = split_length
+        self.split_overlap = split_overlap
+
+    @component.output_types(documents=List[Document])
+    def run(
+        self,
+        document: Document,
+        split_by: Optional[Literal["word", "sentence", "passage"]] = None,
+        split_length: Optional[int] = None,
+        split_overlap: Optional[int] = None,
+    ):
+        """
+        # split the document by split_by after split_length units with an overlap of split_overlap units
+        # return a list of documents with the split texts
+        :param document: the document to split
+        :param split_by: whether to split by word, sentence or passage
+        :param split_length: the maximum number of units in each split
+        :param split_overlap: the number of units that each split should overlap
+        :return: a list of documents with the split texts
+        """
+        if split_by is None:
+            split_by = self.split_by
+        if split_length is None:
+            split_length = self.split_length
+        if split_overlap is None:
+            split_overlap = self.split_overlap
+
+        if document.text is None:
+            raise ValueError("TextDocumentSplitter only works with text documents but document.text is None.")
+        units, split_at = self._split_into_units(document.text, split_by)
+        text_splits = self._concatenate_units(units, split_length, split_overlap, split_at)
+        documents = [Document(text=txt, metadata=document.metadata) for txt in text_splits]
+        return {"documents": documents}
+
+    def to_dict(self) -> Dict[str, Any]:
+        """
+        Serialize this component to a dictionary.
+        """
+        return default_to_dict(
+            self, split_by=self.split_by, split_length=self.split_length, split_overlap=self.split_overlap
+        )
+
+    @classmethod
+    def from_dict(cls, data: Dict[str, Any]) -> "TextDocumentSplitter":
+        """
+        Deserialize this component from a dictionary.
+        """
+        return default_from_dict(cls, data)
+
+    def _split_into_units(self, text: str, split_by: Literal["word", "sentence", "passage"]) -> Tuple[List[str], str]:
+        if split_by == "passage":
+            elements = text.split("\n\n")
+            split_at = "\n\n"
+        elif split_by == "sentence":
+            elements = self._split_sentences(text)
+            split_at = "."
+        elif split_by == "word":
+            elements = text.split(" ")
+            split_at = " "
+        else:
+            raise NotImplementedError("PreProcessor only supports 'passage', 'sentence' or 'word' split_by options.")
+
+        return elements, split_at
+
+    def _concatenate_units(
+        self, elements: List[str], split_length: int, split_overlap: int, split_at: str
+    ) -> List[str]:
+        """
+        Concatenates the elements into parts of split_length units.
+        """
+        text_splits = []
+        segments = windowed(elements, n=split_length, step=split_length - split_overlap)
+        for seg in segments:
+            current_units = [unit for unit in seg if unit is not None]
+            txt = split_at.join(current_units)
+            if len(txt) > 0:
+                text_splits.append(txt)
+        return text_splits
+
+    def _split_sentences(self, text: str) -> List[str]:
+        """
+        Split text into sentences. Naive implementation that splits by ".".
+        :param text: The text to split.
+        :return: The list of sentences.
+        """
+        return text.split(".")
diff --git a/releasenotes/notes/preprocessor-2-0-9828d930562fa3f5.yaml b/releasenotes/notes/preprocessor-2-0-9828d930562fa3f5.yaml
new file mode 100644
index 0000000000..bb58645d6d
--- /dev/null
+++ b/releasenotes/notes/preprocessor-2-0-9828d930562fa3f5.yaml
@@ -0,0 +1,4 @@
+---
+preview:
+  - |
+    Add the `TextDocumentSplitter` component for Haystack 2.0 that splits a Document with long text into multiple Documents with shorter texts. Thereby the texts match the maximum length that the language models in Embedders or other components can process.
diff --git a/test/preview/components/preprocessors/__init__.py b/test/preview/components/preprocessors/__init__.py
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/test/preview/components/preprocessors/test_text_document_splitter.py b/test/preview/components/preprocessors/test_text_document_splitter.py
new file mode 100644
index 0000000000..6c8dec1a8f
--- /dev/null
+++ b/test/preview/components/preprocessors/test_text_document_splitter.py
@@ -0,0 +1,110 @@
+import pytest
+
+from haystack.preview import Document
+from haystack.preview.components.preprocessors import TextDocumentSplitter
+
+
+class TestTextDocumentSplitter:
+    @pytest.mark.unit
+    def test_non_text_document(self):
+        with pytest.raises(
+            ValueError, match="TextDocumentSplitter only works with text documents but document.text is None."
+        ):
+            splitter = TextDocumentSplitter()
+            splitter.run(document=Document())
+
+    @pytest.mark.unit
+    def test_unsupported_split_option(self):
+        with pytest.raises(
+            NotImplementedError, match="PreProcessor only supports 'passage', 'sentence' or 'word' split_by options."
+        ):
+            splitter = TextDocumentSplitter()
+            splitter.run(document=Document(text="text"), split_by="unsupported")
+
+    @pytest.mark.unit
+    def test_split_by_word(self):
+        splitter = TextDocumentSplitter()
+        result = splitter.run(
+            document=Document(
+                text="This is a text with some words. There is a second sentence. And there is a third sentence."
+            ),
+            split_by="word",
+            split_length=10,
+        )
+        assert len(result["documents"]) == 2
+        assert result["documents"][0].text == "This is a text with some words. There is a"
+        assert result["documents"][1].text == "second sentence. And there is a third sentence."
+
+    @pytest.mark.unit
+    def test_split_by_sentence(self):
+        splitter = TextDocumentSplitter()
+        result = splitter.run(
+            document=Document(
+                text="This is a text with some words. There is a second sentence. And there is a third sentence."
+            ),
+            split_by="sentence",
+            split_length=1,
+        )
+        assert len(result["documents"]) == 3
+        assert result["documents"][0].text == "This is a text with some words"
+        assert result["documents"][1].text == " There is a second sentence"
+        assert result["documents"][2].text == " And there is a third sentence"
+
+    @pytest.mark.unit
+    def test_split_by_passage(self):
+        splitter = TextDocumentSplitter()
+        result = splitter.run(
+            document=Document(
+                text="This is a text with some words. There is a second sentence.\n\nAnd there is a third sentence.\n\n And another passage."
+            ),
+            split_by="passage",
+            split_length=1,
+        )
+        assert len(result["documents"]) == 3
+        assert result["documents"][0].text == "This is a text with some words. There is a second sentence."
+        assert result["documents"][1].text == "And there is a third sentence."
+        assert result["documents"][2].text == " And another passage."
+
+    @pytest.mark.unit
+    def test_split_by_word_with_overlap(self):
+        splitter = TextDocumentSplitter()
+        result = splitter.run(
+            document=Document(
+                text="This is a text with some words. There is a second sentence. And there is a third sentence."
+            ),
+            split_by="word",
+            split_length=10,
+            split_overlap=2,
+        )
+        assert len(result["documents"]) == 2
+        assert result["documents"][0].text == "This is a text with some words. There is a"
+        assert result["documents"][1].text == "is a second sentence. And there is a third sentence."
+
+    @pytest.mark.unit
+    def test_to_dict(self):
+        splitter = TextDocumentSplitter()
+        data = splitter.to_dict()
+        assert data == {
+            "type": "TextDocumentSplitter",
+            "init_parameters": {"split_by": "word", "split_length": 200, "split_overlap": 0},
+        }
+
+    @pytest.mark.unit
+    def test_to_dict_with_custom_init_parameters(self):
+        splitter = TextDocumentSplitter(split_by="passage", split_length=100, split_overlap=1)
+        data = splitter.to_dict()
+        assert data == {
+            "type": "TextDocumentSplitter",
+            "init_parameters": {"split_by": "passage", "split_length": 100, "split_overlap": 1},
+        }
+
+    @pytest.mark.unit
+    def test_from_dict(self):
+        data = {
+            "type": "TextDocumentSplitter",
+            "init_parameters": {"split_by": "passage", "split_length": 100, "split_overlap": 1},
+        }
+        splitter = TextDocumentSplitter.from_dict(data)
+        assert splitter.split_by == "passage"
+        assert splitter.split_length == 100
+        assert splitter.split_overlap == 1


Code review comments:
diff_hunk: @@ -0,0 +1,109 @@
+from typing import List, Optional, Dict, Any, Literal, Tuple
+
+from more_itertools import windowed
+
+from haystack.preview import component, Document, default_from_dict, default_to_dict
+
+
+@component
+class TextDocumentSplitter:
+    """
+    Split a text document into a list of text documents with shorter texts.
+    This is useful for splitting documents with long texts that otherwise would not fit into the maximum text length of language models
+    """
+
+    def __init__(
+        self, split_by: Literal["word", "sentence", "passage"] = "word", split_length: int = 200, split_overlap: int = 0
+    ):
+        """
+        :param split_by: The unit by which the document should be split. Choose from "word", "sentence", "passage".
+        :param split_length: The maximum length of each split.
+        :param split_overlap: The number of units that each split should overlap.
+        """
+        self.split_by = split_by
+        self.split_length = split_length
+        self.split_overlap = split_overlap
+
+    @component.output_types(documents=List[Document])
+    def run(
+        self,
+        document: Document,
+        split_by: Optional[Literal["word", "sentence", "passage"]] = None,
+        split_length: Optional[int] = None,
+        split_overlap: Optional[int] = None,
+    ):
+        """
+        # split the document by split_by after split_length units with an overlap of split_overlap units
+        # return a list of documents with the split texts
+        :param document: the document to split
+        :param split_by: whether to split by word, sentence or passage
+        :param split_length: the maximum number of units in each split
+        :param split_overlap: the number of units that each split should overlap
+        :return: a list of documents with the split texts
+        """
+        if split_by is None:
+            split_by = self.split_by
+        if split_length is None:
+            split_length = self.split_length
+        if split_overlap is None:
+            split_overlap = self.split_overlap
+
+        if document.text is None:
+            raise ValueError("TextDocumentSplitter only works with text documents but document.text is None.")
+        units, split_at = self._split_into_units(document.text, split_by)
+        text_splits = self._concatenate_units(units, split_length, split_overlap, split_at)
+        documents = [Document(text=txt, metadata=document.metadata) for txt in text_splits]
+        return {"documents": documents}
+
+    def to_dict(self) -> Dict[str, Any]:
+        """
+        Serialize this component to a dictionary.
+        """
+        return default_to_dict(
+            self, split_by=self.split_by, split_length=self.split_length, split_overlap=self.split_overlap
+        )
+
+    @classmethod
+    def from_dict(cls, data: Dict[str, Any]) -> "TextDocumentSplitter":
+        """
+        Deserialize this component from a dictionary.
+        """
+        return default_from_dict(cls, data)
+
+    def _split_into_units(self, text: str, split_by: Literal["word", "sentence", "passage"]) -> Tuple[List[str], str]:
+        if split_by == "passage":
+            elements = text.split("\n\n")
+            split_at = "\n\n"
+        elif split_by == "sentence":
+            elements = self._split_sentences(text)
+            split_at = "."
+        elif split_by == "word":
+            elements = text.split(" ")
+            split_at = " "
+        else:
+            raise NotImplementedError("PreProcessor only supports 'passage', 'sentence' or 'word' split_by options.")
+
+        return elements, split_at
path: haystack/preview/components/preprocessors/text_document_splitter.py
text: @user1:
Minor: we can return directly.

```suggestion
        if split_by == "passage":
            return text.split("\n\n"), "\n\n"
        if split_by == "sentence":
            return self._split_sentences(text), "."
        if split_by == "word":
            return text.split(" "), " "
        raise NotImplementedError("PreProcessor only supports 'passage', 'sentence' or 'word' split_by options.")
```

diff_hunk: @@ -0,0 +1,109 @@
+from typing import List, Optional, Dict, Any, Literal, Tuple
+
+from more_itertools import windowed
+
+from haystack.preview import component, Document, default_from_dict, default_to_dict
+
+
+@component
+class TextDocumentSplitter:
+    """
+    Split a text document into a list of text documents with shorter texts.
+    This is useful for splitting documents with long texts that otherwise would not fit into the maximum text length of language models
+    """
+
+    def __init__(
+        self, split_by: Literal["word", "sentence", "passage"] = "word", split_length: int = 200, split_overlap: int = 0
+    ):
+        """
+        :param split_by: The unit by which the document should be split. Choose from "word", "sentence", "passage".
path: haystack/preview/components/preprocessors/text_document_splitter.py
text: @user1:
Let's clarify the meaning of these three values and their limitations, i.e.: `word` implies splitting by whitespace, `sentence` splitting by dot and `passage`/`paragraph` splits by double-return

diff_hunk: @@ -0,0 +1,109 @@
+from typing import List, Optional, Dict, Any, Literal, Tuple
+
+from more_itertools import windowed
+
+from haystack.preview import component, Document, default_from_dict, default_to_dict
+
+
+@component
+class TextDocumentSplitter:
+    """
+    Split a text document into a list of text documents with shorter texts.
+    This is useful for splitting documents with long texts that otherwise would not fit into the maximum text length of language models
+    """
+
+    def __init__(
+        self, split_by: Literal["word", "sentence", "passage"] = "word", split_length: int = 200, split_overlap: int = 0
+    ):
+        """
+        :param split_by: The unit by which the document should be split. Choose from "word", "sentence", "passage".
+        :param split_length: The maximum length of each split.
+        :param split_overlap: The number of units that each split should overlap.
+        """
+        self.split_by = split_by
+        self.split_length = split_length
+        self.split_overlap = split_overlap
+
+    @component.output_types(documents=List[Document])
+    def run(
+        self,
+        document: Document,
+        split_by: Optional[Literal["word", "sentence", "passage"]] = None,
+        split_length: Optional[int] = None,
+        split_overlap: Optional[int] = None,
+    ):
+        """
+        # split the document by split_by after split_length units with an overlap of split_overlap units
+        # return a list of documents with the split texts
+        :param document: the document to split
+        :param split_by: whether to split by word, sentence or passage
+        :param split_length: the maximum number of units in each split
+        :param split_overlap: the number of units that each split should overlap
+        :return: a list of documents with the split texts
+        """
+        if split_by is None:
+            split_by = self.split_by
+        if split_length is None:
+            split_length = self.split_length
+        if split_overlap is None:
+            split_overlap = self.split_overlap
+
+        if document.text is None:
+            raise ValueError("TextDocumentSplitter only works with text documents but document.text is None.")
+        units, split_at = self._split_into_units(document.text, split_by)
+        text_splits = self._concatenate_units(units, split_length, split_overlap, split_at)
+        documents = [Document(text=txt, metadata=document.metadata) for txt in text_splits]
+        return {"documents": documents}
+
+    def to_dict(self) -> Dict[str, Any]:
+        """
+        Serialize this component to a dictionary.
+        """
+        return default_to_dict(
+            self, split_by=self.split_by, split_length=self.split_length, split_overlap=self.split_overlap
+        )
+
+    @classmethod
+    def from_dict(cls, data: Dict[str, Any]) -> "TextDocumentSplitter":
+        """
+        Deserialize this component from a dictionary.
+        """
+        return default_from_dict(cls, data)
+
+    def _split_into_units(self, text: str, split_by: Literal["word", "sentence", "passage"]) -> Tuple[List[str], str]:
+        if split_by == "passage":
+            elements = text.split("\n\n")
+            split_at = "\n\n"
+        elif split_by == "sentence":
+            elements = self._split_sentences(text)
+            split_at = "."
+        elif split_by == "word":
+            elements = text.split(" ")
+            split_at = " "
+        else:
+            raise NotImplementedError("PreProcessor only supports 'passage', 'sentence' or 'word' split_by options.")
+
+        return elements, split_at
+
+    def _concatenate_units(
+        self, elements: List[str], split_length: int, split_overlap: int, split_at: str
+    ) -> List[str]:
+        """
+        Concatenates the elements into parts of split_length units.
+        """
+        text_splits = []
+        segments = windowed(elements, n=split_length, step=split_length - split_overlap)
+        for seg in segments:
+            current_units = [unit for unit in seg if unit is not None]
+            txt = split_at.join(current_units)
+            if len(txt) > 0:
+                text_splits.append(txt)
+        return text_splits
+
+    def _split_sentences(self, text: str) -> List[str]:
+        """
+        Split text into sentences. Naive implementation that splits by ".".
+        :param text: The text to split.
+        :return: The list of sentences.
+        """
+        return text.split(".")
path: haystack/preview/components/preprocessors/text_document_splitter.py
text: @user1:
I wonder if we even need a method for this. I know that this is just a placeholder for a better function that we can implement later, but if we already plan to replace it we can just remove it.

@author:
Removed it for now to keep it as simple as possible. We'll certainly need to re-add it later.

diff_hunk: @@ -0,0 +1,109 @@
+from typing import List, Optional, Dict, Any, Literal, Tuple
+
+from more_itertools import windowed
+
+from haystack.preview import component, Document, default_from_dict, default_to_dict
+
+
+@component
+class TextDocumentSplitter:
+    """
+    Split a text document into a list of text documents with shorter texts.
+    This is useful for splitting documents with long texts that otherwise would not fit into the maximum text length of language models
+    """
+
+    def __init__(
+        self, split_by: Literal["word", "sentence", "passage"] = "word", split_length: int = 200, split_overlap: int = 0
+    ):
+        """
+        :param split_by: The unit by which the document should be split. Choose from "word", "sentence", "passage".
+        :param split_length: The maximum length of each split.
+        :param split_overlap: The number of units that each split should overlap.
+        """
+        self.split_by = split_by
+        self.split_length = split_length
+        self.split_overlap = split_overlap
+
+    @component.output_types(documents=List[Document])
+    def run(
+        self,
+        document: Document,
path: haystack/preview/components/preprocessors/text_document_splitter.py
text: @user1:
I am really trying not to contradict myself too much on this topic, but: let's take a list of Documents here :see_no_evil: 

The fact is that I don't see almost any scenario where this component can be used on a single document. Normally this component is expected to process a list of documents coming from one or more file converters, which all output lists (see any of the components here: https://github.com/deepset-ai/haystack/tree/main/haystack/preview/components/file_converters).

It's also possible that the converters are all wrong! They've been made one from the blueprint of the other. However, in order to connect this with a converter we need to either change this input to a list or we need to change the converters ouput. In turn, changing the converters requires us to change the LinkContentFetcher, which in turn means that it's hard to connect it with the WebSearch component...

Note that making this component take a list as input does *not* affect the output type. That stays a flat list.

Happy to discuss this offline too. There might be a better solution to this problem.

@author:
Alright, List[Document] is now the input and output type. Still having the concept of indexing pipelines in mind, that makes more sense then single Document as input and we can introduce support for single Document etc. later with a couple of different options.

diff_hunk: @@ -0,0 +1,109 @@
+from typing import List, Optional, Dict, Any, Literal, Tuple
+
+from more_itertools import windowed
+
+from haystack.preview import component, Document, default_from_dict, default_to_dict
+
+
+@component
+class TextDocumentSplitter:
+    """
+    Split a text document into a list of text documents with shorter texts.
+    This is useful for splitting documents with long texts that otherwise would not fit into the maximum text length of language models
+    """
+
+    def __init__(
+        self, split_by: Literal["word", "sentence", "passage"] = "word", split_length: int = 200, split_overlap: int = 0
+    ):
+        """
+        :param split_by: The unit by which the document should be split. Choose from "word", "sentence", "passage".
+        :param split_length: The maximum length of each split.
+        :param split_overlap: The number of units that each split should overlap.
path: haystack/preview/components/preprocessors/text_document_splitter.py
text: @user1:
This is an ambiguity I also found in the old PreProcessor (I haven't read the code yet): what is the unit we're taling about?

For example: if I set `split_by=sentence, split_length=3`, will the splitter give me three sentence long chunks? 

I believe the old PreProcessor always interpreted the unit to be a word, but I believe that was quite confusing. Let's clarify what is the situation here, so regardless whether people knew how the old PreProcessor worked, the new behavior is clear.

@author:
The unit is what is set in `split_by`. I extended the description to make that more clear. In your example, the splitter will return Documents containing three sentences each, yes.
The old preprocessor used also `split_by` as the unit for `split_length`. What I saw is that the description is confusing though as it says "split_overlap: **Word** overlap between two adjacent documents after a split." where it should be again unit not word.

diff_hunk: @@ -0,0 +1,109 @@
+from typing import List, Optional, Dict, Any, Literal, Tuple
+
+from more_itertools import windowed
+
+from haystack.preview import component, Document, default_from_dict, default_to_dict
+
+
+@component
+class TextDocumentSplitter:
+    """
+    Split a text document into a list of text documents with shorter texts.
+    This is useful for splitting documents with long texts that otherwise would not fit into the maximum text length of language models
+    """
+
+    def __init__(
+        self, split_by: Literal["word", "sentence", "passage"] = "word", split_length: int = 200, split_overlap: int = 0
+    ):
+        """
+        :param split_by: The unit by which the document should be split. Choose from "word", "sentence", "passage".
+        :param split_length: The maximum length of each split.
+        :param split_overlap: The number of units that each split should overlap.
+        """
+        self.split_by = split_by
+        self.split_length = split_length
+        self.split_overlap = split_overlap
+
+    @component.output_types(documents=List[Document])
+    def run(
+        self,
+        document: Document,
+        split_by: Optional[Literal["word", "sentence", "passage"]] = None,
+        split_length: Optional[int] = None,
+        split_overlap: Optional[int] = None,
path: haystack/preview/components/preprocessors/text_document_splitter.py
text: @user1:
I have mixed feelings about these :sweat_smile: How frequent is it that these values get changed at runtime? I'd rather add them later than having them here unused

@author:
Let's leave it out then to keep this as simple as possible. I don't expect this to limit users in their use cases.