Repository: deepset-ai/haystack

Issue:
OpenAI errors not caught
**Describe the bug**
When using the openai API a lot, e.g. during experiments, I get a lot of errors that are not caught.

In the previous haystack version we caught all OpenAIError, now it is just the OpenAIRateLimitError, see https://github.com/deepset-ai/haystack/pull/4460/files#r1182887215


**Error message**
File "/opt/homebrew/Caskroom/miniforge/base/envs/haystack/lib/python3.9/site-packages/openai/api_requestor.py", line 527, in request_raw
    raise error.Timeout("Request timed out: {}".format(e)) from e
openai.error.Timeout: Request timed out: HTTPSConnectionPool(host='api.openai.com', port=443): Read timed out. (read timeout=600)

Process finished with exit code 1

**Expected behavior**
since the openAI API is quite unstable I would love to catch those errors.


Discussion:

Resolving patch:
diff --git a/haystack/utils/openai_utils.py b/haystack/utils/openai_utils.py
--- a/haystack/utils/openai_utils.py
+++ b/haystack/utils/openai_utils.py
@@ -128,7 +128,9 @@ def _openai_text_completion_tokenization_details(model_name: str):
 
 
 @tenacity.retry(
-    retry=tenacity.retry_if_exception_type(OpenAIRateLimitError),
+    reraise=True,
+    retry=tenacity.retry_if_exception_type(OpenAIError)
+    and tenacity.retry_if_not_exception_type(OpenAIUnauthorizedError),
     wait=tenacity.wait_exponential(multiplier=OPENAI_BACKOFF),
     stop=tenacity.stop_after_attempt(OPENAI_MAX_RETRIES),
 )


Test patch:
diff --git a/test/utils/test_openai_utils.py b/test/utils/test_openai_utils.py
new file mode 100644
--- /dev/null
+++ b/test/utils/test_openai_utils.py
@@ -0,0 +1,57 @@
+from unittest.mock import patch
+
+import pytest
+from tenacity import wait_none
+
+from haystack.errors import OpenAIError, OpenAIRateLimitError, OpenAIUnauthorizedError
+from haystack.utils.openai_utils import openai_request
+
+
+@pytest.mark.unit
+@patch("haystack.utils.openai_utils.requests")
+def test_openai_request_retries_generic_error(mock_requests):
+    mock_requests.request.return_value.status_code = 418
+
+    with pytest.raises(OpenAIError):
+        # We need to use a custom wait amount otherwise the test would take forever to run
+        # as the original wait time is exponential
+        openai_request.retry_with(wait=wait_none())(url="some_url", headers={}, payload={}, read_response=False)
+
+    assert mock_requests.request.call_count == 5
+
+
+@pytest.mark.unit
+@patch("haystack.utils.openai_utils.requests")
+def test_openai_request_retries_on_rate_limit_error(mock_requests):
+    mock_requests.request.return_value.status_code = 429
+
+    with pytest.raises(OpenAIRateLimitError):
+        # We need to use a custom wait amount otherwise the test would take forever to run
+        # as the original wait time is exponential
+        openai_request.retry_with(wait=wait_none())(url="some_url", headers={}, payload={}, read_response=False)
+
+    assert mock_requests.request.call_count == 5
+
+
+@pytest.mark.unit
+@patch("haystack.utils.openai_utils.requests")
+def test_openai_request_does_not_retry_on_unauthorized_error(mock_requests):
+    mock_requests.request.return_value.status_code = 401
+
+    with pytest.raises(OpenAIUnauthorizedError):
+        # We need to use a custom wait amount otherwise the test would take forever to run
+        # as the original wait time is exponential
+        openai_request.retry_with(wait=wait_none())(url="some_url", headers={}, payload={}, read_response=False)
+
+    assert mock_requests.request.call_count == 1
+
+
+@pytest.mark.unit
+@patch("haystack.utils.openai_utils.requests")
+def test_openai_request_does_not_retry_on_success(mock_requests):
+    mock_requests.request.return_value.status_code = 200
+    # We need to use a custom wait amount otherwise the test would take forever to run
+    # as the original wait time is exponential
+    openai_request.retry_with(wait=wait_none())(url="some_url", headers={}, payload={}, read_response=False)
+
+    assert mock_requests.request.call_count == 1