Repository: deepset-ai/haystack

Issue:
"Token indices sequence length is longer than the specified maximum sequence length for this model" with Cohere Command
**Describe the bug**

With a large prompt (>1024 tokens), I get the error

```
Token indices sequence length is longer than the specified maximum sequence length for this model (2710 > 1024). Running this sequence through the model will result in indexing errors
```

when using `PromptNode` with the Cohere "command"  model.

There should be no error, because [the command model supports up to 4096 tokens](https://docs.cohere.com/docs/models).
**Error message**

Token indices sequence length is longer than the specified maximum sequence length for this model (2710 > 1024). Running this sequence through the model will result in indexing errors

**Expected behavior**
No errors or warnings.


**To Reproduce**
```python
import os
from haystack.nodes import PromptNode
from haystack import Document


doc = Document("Haystack is an open-source LLM framework to build production-ready applications. " * 100)
COHERE_API_KEY = os.environ.get("COHERE_API_KEY", None)
pn = PromptNode(model_name_or_path="command", api_key=COHERE_API_KEY)
result = pn.prompt(prompt_template="deepset/question-answering", query="What company is the article about?", documents=[doc])
print(result)

```
**FAQ Check**
- [ X ] Have you had a look at [our new FAQ page](https://docs.haystack.deepset.ai/docs/faq)?

**System:**
 - OS: Ubuntu 22.04
 - Haystack version (commit or version number): 1.19.0


Discussion:
Thanks for raising this issue @parviste! I investigated it and found it that this warning message is raised by the `Tokenizer` inititalized [here](https://github.com/deepset-ai/haystack/blob/011baf492f478f6fb317b3d2ba987261ccacb147/haystack/nodes/prompt/invocation_layer/handlers.py#L65). We don't pass `model_max_length` to the Tokenizer, that's why it assumes that the maximum sequence length is 1024 instead of 4096. I'll create a PR to fix this.

Resolving patch:
diff --git a/haystack/nodes/prompt/invocation_layer/handlers.py b/haystack/nodes/prompt/invocation_layer/handlers.py
--- a/haystack/nodes/prompt/invocation_layer/handlers.py
+++ b/haystack/nodes/prompt/invocation_layer/handlers.py
@@ -63,6 +63,7 @@ class DefaultPromptHandler:
 
     def __init__(self, model_name_or_path: str, model_max_length: int, max_length: int = 100):
         self.tokenizer = AutoTokenizer.from_pretrained(model_name_or_path)
+        self.tokenizer.model_max_length = model_max_length
         self.model_max_length = model_max_length
         self.max_length = max_length
 


Test patch:
diff --git a/test/prompt/test_handlers.py b/test/prompt/test_handlers.py
--- a/test/prompt/test_handlers.py
+++ b/test/prompt/test_handlers.py
@@ -57,6 +57,13 @@ def test_prompt_handler_negative():
     }
 
 
+@pytest.mark.unit
+@patch("haystack.nodes.prompt.invocation_layer.handlers.AutoTokenizer.from_pretrained")
+def test_prompt_handler_model_max_length_set_in_tokenizer(mock_tokenizer):
+    prompt_handler = DefaultPromptHandler(model_name_or_path="model_path", model_max_length=10, max_length=3)
+    assert prompt_handler.tokenizer.model_max_length == 10
+
+
 @pytest.mark.integration
 def test_prompt_handler_basics():
     handler = DefaultPromptHandler(model_name_or_path="gpt2", model_max_length=20, max_length=10)
@@ -65,6 +72,9 @@ def test_prompt_handler_basics():
     handler = DefaultPromptHandler(model_name_or_path="gpt2", model_max_length=20)
     assert handler.max_length == 100
 
+    # test model_max_length is set in tokenizer
+    assert handler.tokenizer.model_max_length == 20
+
 
 @pytest.mark.integration
 def test_gpt2_prompt_handler():