Repository: deepset-ai/haystack

Issue:
PromptNode does't support 16,385 tokens for new gpt-3.5-turbo-1106
**Describe the bug**
Prompt Node is setting a limit of 4096 total tokens on the new gpt-3.5-turbo-1106 model. 

**Error message**
If the max_length is set to 4096, this truncates the prompt to 0 allowed tokens. 
i.e If max_length: 4096
\nException while running node 'PromptNode': No prompt provided. Model gpt-3.5-turbo-1106 requires prompt.Make sure to provide prompt in kwargs.\nEnable debug logging to see the data that was passed when the pipeline failed."

**Expected behavior**
gpt-3.5-turbo-1106 should permit a total context window of 16,385 tokens, and a maximum of 4096 tokens as output. In other words when max_length: 4096, the prompt should allow a maximum of '12289 tokens' as opposed to '0' as it is at the moment. 


**To Reproduce**
Using model gpt-3.5-turbo-1106 in Prompt node set max_length to 4096 and observe a 0 token allowance for the prompt. 


**System:**
 - Haystack version (commit or version number): 1.23


Discussion:

Patch:
diff --git a/haystack/utils/openai_utils.py b/haystack/utils/openai_utils.py
index 25de13e5..c77e0c65 100644
--- a/haystack/utils/openai_utils.py
+++ b/haystack/utils/openai_utils.py
@@ -82,6 +82,9 @@ def _openai_text_completion_tokenization_details(model_name: str):
         elif model_name.startswith("gpt-3.5-turbo-16k") or model_name.startswith("gpt-35-turbo-16k"):
             max_tokens_limit = 16384
             tokenizer_name = model_tokenizer
+        elif model_name.startswith("gpt-3.5-turbo-1106") or model_name.startswith("gpt-35-turbo-1106"):
+            max_tokens_limit = 16384
+            tokenizer_name = model_tokenizer
         elif model_name.startswith("gpt-3"):
             max_tokens_limit = 4096
             tokenizer_name = model_tokenizer
diff --git a/releasenotes/notes/support-gpt-3.5-turbo-1106-6de58fee16cc0c0a.yaml b/releasenotes/notes/support-gpt-3.5-turbo-1106-6de58fee16cc0c0a.yaml
new file mode 100644
index 00000000..92750805
--- /dev/null
+++ b/releasenotes/notes/support-gpt-3.5-turbo-1106-6de58fee16cc0c0a.yaml
@@ -0,0 +1,4 @@
+---
+enhancements:
+  - |
+    Correctly calculate the max token limit for gpt-3.5-turbo-1106
diff --git a/rest_api/pyproject.toml b/rest_api/pyproject.toml
index e5eda60e..2ee766c2 100644
--- a/rest_api/pyproject.toml
+++ b/rest_api/pyproject.toml
@@ -26,7 +26,7 @@ classifiers = [
 ]
 dependencies = [
     "farm-haystack",
-    "fastapi",
+    "fastapi~=0.108.0",
     "uvicorn<1",
     "python-multipart<1",  # optional FastAPI dependency for form data
     "pynvml",


Test patch:
diff --git a/test/utils/test_openai_utils.py b/test/utils/test_openai_utils.py
index 92add5f2..3380a09e 100644
--- a/test/utils/test_openai_utils.py
+++ b/test/utils/test_openai_utils.py
@@ -41,6 +41,13 @@ def test_openai_text_completion_tokenization_details_gpt3_5():
     assert max_tokens_limit == 4096
 
 
+@pytest.mark.unit
+def test_openai_text_completion_tokenization_details_gpt3_5_1106():
+    tokenizer_name, max_tokens_limit = _openai_text_completion_tokenization_details(model_name="gpt-3.5-turbo-1106")
+    assert tokenizer_name == "cl100k_base"
+    assert max_tokens_limit == 16384
+
+
 @pytest.mark.unit
 def test_openai_text_completion_tokenization_details_gpt_4():
     tokenizer_name, max_tokens_limit = _openai_text_completion_tokenization_details(model_name="gpt-4")