Repository: huggingface/huggingface_hub
Pull request #1935 (2023-12-21 15:27:39)

Raise on failed inference endpoint in `InferenceEndpoint.wait()`
_Originally from @philschmid [on slack](https://huggingface.slack.com/archives/C02V5EA0A95/p1703081113214549) (internal):_

> i am working on a IE example with the new version of huggingface_hub which supports custom container image configs. I noticed that .wait will not break if the endpoint failed.
I made a mistake an the endpoint is in the "failed" status through the UI but the endpoint.wait() is still "blocking".

We now raise an error if the inference endpoint gets into a `failed` state instead of waiting forever.

HuggingFaceDocBuilderDev: The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/huggingface_hub/pr_1935). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
codecov[bot]: ## [Codecov](https://app.codecov.io/gh/huggingface/huggingface_hub/pull/1935?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Comparison is base [(`7c2aac6`)](https://app.codecov.io/gh/huggingface/huggingface_hub/commit/7c2aac6b06f623d06e9a5b5e5c434b39256f7d65?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface) 82.44% compared to head [(`456159f`)](https://app.codecov.io/gh/huggingface/huggingface_hub/pull/1935?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface) 82.42%.
> Report is 1 commits behind head on main.


<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##             main    #1935      +/-   ##
==========================================
- Coverage   82.44%   82.42%   -0.03%     
==========================================
  Files          66       66              
  Lines        8138     8140       +2     
==========================================
  Hits         6709     6709              
- Misses       1429     1431       +2     
```



</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/huggingface/huggingface_hub/pull/1935?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface).

Wauplin: Thanks for reviewing!

Patch:
diff --git a/src/huggingface_hub/_inference_endpoints.py b/src/huggingface_hub/_inference_endpoints.py
index 291e2836..90b9d324 100644
--- a/src/huggingface_hub/_inference_endpoints.py
+++ b/src/huggingface_hub/_inference_endpoints.py
@@ -192,6 +192,12 @@ class InferenceEndpoint:
 
         Returns:
             [`InferenceEndpoint`]: the same Inference Endpoint, mutated in place with the latest data.
+
+        Raises:
+            [`InferenceEndpointError`]
+                If the Inference Endpoint ended up in a failed state.
+            [`InferenceEndpointTimeoutError`]
+                If the Inference Endpoint is not deployed after `timeout` seconds.
         """
         if self.url is not None:  # Means the endpoint is deployed
             logger.info("Inference Endpoint is ready to be used.")
@@ -208,6 +214,10 @@ class InferenceEndpoint:
             if self.url is not None:  # Means the endpoint is deployed
                 logger.info("Inference Endpoint is ready to be used.")
                 return self
+            if self.status == InferenceEndpointStatus.FAILED:
+                raise InferenceEndpointError(
+                    f"Inference Endpoint {self.name} failed to deploy. Please check the logs for more information."
+                )
             if timeout is not None:
                 if time.time() - start > timeout:
                     raise InferenceEndpointTimeoutError("Timeout while waiting for Inference Endpoint to be deployed.")


Test patch:
diff --git a/tests/test_inference_endpoints.py b/tests/test_inference_endpoints.py
index 66da6740..319535ba 100644
--- a/tests/test_inference_endpoints.py
+++ b/tests/test_inference_endpoints.py
@@ -76,6 +76,37 @@ MOCK_RUNNING = {
     },
 }
 
+MOCK_FAILED = {
+    "name": "my-endpoint-name",
+    "type": "protected",
+    "accountId": None,
+    "provider": {"vendor": "aws", "region": "us-east-1"},
+    "compute": {
+        "accelerator": "cpu",
+        "instanceType": "c6i",
+        "instanceSize": "medium",
+        "scaling": {"minReplica": 0, "maxReplica": 1},
+    },
+    "model": {
+        "repository": "gpt2",
+        "revision": "11c5a3d5811f50298f278a704980280950aedb10",
+        "task": "text-generation",
+        "framework": "pytorch",
+        "image": {"huggingface": {}},
+    },
+    "status": {
+        "createdAt": "2023-10-26T12:41:53.263Z",
+        "createdBy": {"id": "6273f303f6d63a28483fde12", "name": "Wauplin"},
+        "updatedAt": "2023-10-26T12:41:53.263Z",
+        "updatedBy": {"id": "6273f303f6d63a28483fde12", "name": "Wauplin"},
+        "private": None,
+        "state": "failed",
+        "message": "Endpoint failed to deploy",
+        "readyReplica": 0,
+        "targetReplica": 1,
+    },
+}
+
 
 def test_from_raw_initialization():
     """Test InferenceEndpoint is correctly initialized from raw dict."""
@@ -188,6 +219,20 @@ def test_wait_timeout(mock_get: Mock):
     assert len(mock_get.call_args_list) == 3
 
 
+@patch("huggingface_hub.hf_api.HfApi.get_inference_endpoint")
+def test_wait_failed(mock_get: Mock):
+    """Test waits until timeout error is raised."""
+    endpoint = InferenceEndpoint.from_raw(MOCK_INITIALIZING, namespace="foo")
+
+    mock_get.side_effect = [
+        InferenceEndpoint.from_raw(MOCK_INITIALIZING, namespace="foo"),
+        InferenceEndpoint.from_raw(MOCK_INITIALIZING, namespace="foo"),
+        InferenceEndpoint.from_raw(MOCK_FAILED, namespace="foo"),
+    ]
+    with pytest.raises(InferenceEndpointError, match=".*failed to deploy.*"):
+        endpoint.wait(refresh_every=0.001)
+
+
 @patch("huggingface_hub.hf_api.HfApi.pause_inference_endpoint")
 def test_pause(mock: Mock):
     """Test `pause` calls the correct alias."""