Repository: huggingface/huggingface_hub
Pull request #1606 (2023-08-20 11:55:46)

Add text classification to inference client
**Add  text-classification to HuggingFace🤗 Hub**

References https://github.com/huggingface/huggingface_hub/issues/1539

This is an ongoing list of model tasks to implement in the Hugging Face Hub inference client. Each task is planned to be its own PR. The task for this is [text-classification](https://huggingface.co/tasks/text-classification).

**Key Features**

- Modifies _client.py to call a text classification model.
- Modifies testing and documentation to reflect changes.

martinbrose: > Hey @martinbrose, I'm starting to review all your inference-related PRs. Thanks a ton for the massive work, it's good quality for what I saw! 👏 🙏 I have left some comments on this PR that are also relevant to the other PRs (especially simplifying as much as possible the methods signature + the merge conflict issue). In the meantime, I'll take the time to thoroughly review the other PRs.
> 
> FYI, I'm off this Thursday/Friday and be fully back on the project starting from next week :)

Thanks for the review!
Apologies for this long list of separate PRs. I started with one... and then couldn't stop using them as bite-sized challenges at night.
codecov[bot]: ## [Codecov](https://app.codecov.io/gh/huggingface/huggingface_hub/pull/1606?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface) Report
Patch coverage: **`62.50%`** and project coverage change: **`-0.35%`** :warning:
> Comparison is base [(`94c427a`)](https://app.codecov.io/gh/huggingface/huggingface_hub/commit/94c427aa236ec43dfd1ef83e225e4ee8242b04e0?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface) 82.13% compared to head [(`8ed6dd9`)](https://app.codecov.io/gh/huggingface/huggingface_hub/pull/1606?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface) 81.78%.

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##             main    #1606      +/-   ##
==========================================
- Coverage   82.13%   81.78%   -0.35%     
==========================================
  Files          60       60              
  Lines        6746     6754       +8     
==========================================
- Hits         5541     5524      -17     
- Misses       1205     1230      +25     
```


| [Files Changed](https://app.codecov.io/gh/huggingface/huggingface_hub/pull/1606?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface) | Coverage Δ | |
|---|---|---|
| [...gingface\_hub/inference/\_generated/\_async\_client.py](https://app.codecov.io/gh/huggingface/huggingface_hub/pull/1606?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface#diff-c3JjL2h1Z2dpbmdmYWNlX2h1Yi9pbmZlcmVuY2UvX2dlbmVyYXRlZC9fYXN5bmNfY2xpZW50LnB5) | `58.37% <25.00%> (-0.62%)` | :arrow_down: |
| [src/huggingface\_hub/inference/\_client.py](https://app.codecov.io/gh/huggingface/huggingface_hub/pull/1606?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface#diff-c3JjL2h1Z2dpbmdmYWNlX2h1Yi9pbmZlcmVuY2UvX2NsaWVudC5weQ==) | `79.51% <100.00%> (+0.40%)` | :arrow_up: |

... and [6 files with indirect coverage changes](https://app.codecov.io/gh/huggingface/huggingface_hub/pull/1606/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface)


</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/huggingface/huggingface_hub/pull/1606?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface).

Wauplin: I've merged the suggested changes (see above) and tried it locally. It works great! :)
Will merge the PR once CI is green.

Patch:
diff --git a/docs/source/en/guides/inference.md b/docs/source/en/guides/inference.md
index 5ee7d7a1..ee01fc1d 100644
--- a/docs/source/en/guides/inference.md
+++ b/docs/source/en/guides/inference.md
@@ -139,7 +139,7 @@ has a simple API that supports the most common tasks. Here is a list of the curr
 | | [Sentence Similarity](https://huggingface.co/tasks/sentence-similarity)            | ✅ | [`~InferenceClient.sentence_similarity`] |
 | | [Summarization](https://huggingface.co/tasks/summarization)                  | ✅ | [`~InferenceClient.summarization`] |
 | | [Table Question Answering](https://huggingface.co/tasks/table-question-answering)       | |  |
-| | [Text Classification](https://huggingface.co/tasks/text-classification)            | |  |
+| | [Text Classification](https://huggingface.co/tasks/text-classification)            | ✅ | [`~InferenceClient.text_classification`] |
 | | [Text Generation](https://huggingface.co/tasks/text-generation)   | ✅ | [`~InferenceClient.text_generation`] |
 | | [Token Classification](https://huggingface.co/tasks/token-classification)           | |  |
 | | [Translation](https://huggingface.co/tasks/translation)       | |  |
diff --git a/src/huggingface_hub/inference/_client.py b/src/huggingface_hub/inference/_client.py
index 308e1739..be04edc3 100644
--- a/src/huggingface_hub/inference/_client.py
+++ b/src/huggingface_hub/inference/_client.py
@@ -765,6 +765,38 @@ class InferenceClient:
         response = self.post(json=payload, model=model, task="summarization")
         return _bytes_to_dict(response)[0]["summary_text"]
 
+    def text_classification(self, text: str, *, model: Optional[str] = None) -> List[ClassificationOutput]:
+        """
+        Perform sentiment-analysis on the given text.
+
+        Args:
+            text (`str`):
+                A string to be classified.
+            model (`str`, *optional*):
+                The model to use for the text classification task. Can be a model ID hosted on the Hugging Face Hub or a URL to
+                a deployed Inference Endpoint. If not provided, the default recommended text classification model will be used.
+                Defaults to None.
+
+        Returns:
+            `List[Dict]`: a list of dictionaries containing the predicted label and associated probability.
+
+        Raises:
+            [`InferenceTimeoutError`]:
+                If the model is unavailable or the request times out.
+            `HTTPError`:
+                If the request fails with an HTTP error status code other than HTTP 503.
+
+        Example:
+        ```py
+        >>> from huggingface_hub import InferenceClient
+        >>> client = InferenceClient()
+        >>> output = client.text_classification("I like you")
+        [{'label': 'POSITIVE', 'score': 0.9998695850372314}, {'label': 'NEGATIVE', 'score': 0.0001304351753788069}]
+        ```
+        """
+        response = self.post(json={"inputs": text}, model=model, task="text-classification")
+        return _bytes_to_list(response)[0]
+
     @overload
     def text_generation(  # type: ignore
         self,
diff --git a/src/huggingface_hub/inference/_generated/_async_client.py b/src/huggingface_hub/inference/_generated/_async_client.py
index 90629a99..70341d6d 100644
--- a/src/huggingface_hub/inference/_generated/_async_client.py
+++ b/src/huggingface_hub/inference/_generated/_async_client.py
@@ -772,6 +772,39 @@ class AsyncInferenceClient:
         response = await self.post(json=payload, model=model, task="summarization")
         return _bytes_to_dict(response)[0]["summary_text"]
 
+    async def text_classification(self, text: str, *, model: Optional[str] = None) -> List[ClassificationOutput]:
+        """
+        Perform sentiment-analysis on the given text.
+
+        Args:
+            text (`str`):
+                A string to be classified.
+            model (`str`, *optional*):
+                The model to use for the text classification task. Can be a model ID hosted on the Hugging Face Hub or a URL to
+                a deployed Inference Endpoint. If not provided, the default recommended text classification model will be used.
+                Defaults to None.
+
+        Returns:
+            `List[Dict]`: a list of dictionaries containing the predicted label and associated probability.
+
+        Raises:
+            [`InferenceTimeoutError`]:
+                If the model is unavailable or the request times out.
+            `aiohttp.ClientResponseError`:
+                If the request fails with an HTTP error status code other than HTTP 503.
+
+        Example:
+        ```py
+        # Must be run in an async context
+        >>> from huggingface_hub import AsyncInferenceClient
+        >>> client = AsyncInferenceClient()
+        >>> output = await client.text_classification("I like you")
+        [{'label': 'POSITIVE', 'score': 0.9998695850372314}, {'label': 'NEGATIVE', 'score': 0.0001304351753788069}]
+        ```
+        """
+        response = await self.post(json={"inputs": text}, model=model, task="text-classification")
+        return _bytes_to_list(response)[0]
+
     @overload
     async def text_generation(  # type: ignore
         self,


Test patch:
diff --git a/tests/cassettes/InferenceClientVCRTest.test_text_classification.yaml b/tests/cassettes/InferenceClientVCRTest.test_text_classification.yaml
new file mode 100644
index 00000000..67e8ac80
--- /dev/null
+++ b/tests/cassettes/InferenceClientVCRTest.test_text_classification.yaml
@@ -0,0 +1,48 @@
+interactions:
+- request:
+    body: '{"inputs": ["I like you"]}'
+    headers:
+      Accept:
+      - '*/*'
+      Accept-Encoding:
+      - gzip, deflate, br
+      Connection:
+      - keep-alive
+      Content-Length:
+      - '41'
+      Content-Type:
+      - application/json
+      X-Amzn-Trace-Id:
+      - b658f44b-c82c-4a0c-9fc1-c287ea0b66d3
+      user-agent:
+      - unknown/None; hf_hub/0.17.0.dev0; python/3.10.12
+    method: POST
+    uri: https://api-inference.huggingface.co/models/distilbert-base-uncased-finetuned-sst-2-english
+  response:
+    body:
+      string: '[[{"label":"POSITIVE","score":0.9998695850372314},{"label":"NEGATIVE","score":0.0001304351753788069}]]'
+    headers:
+      Connection:
+      - keep-alive
+      Content-Length:
+      - '204'
+      Content-Type:
+      - application/json
+      Date:
+      - Sun, 20 Aug 2023 11:48:55 GMT
+      access-control-allow-credentials:
+      - 'true'
+      vary:
+      - Origin, Access-Control-Request-Method, Access-Control-Request-Headers
+      x-compute-time:
+      - '0.033'
+      x-compute-type:
+      - cache
+      x-request-id:
+      - MiuTWky1u3OlV7JlitniT
+      x-sha:
+      - 3d65bad49c7ba6f71920504507a8927f4b9db6c0
+    status:
+      code: 200
+      message: OK
+version: 1
diff --git a/tests/test_inference_client.py b/tests/test_inference_client.py
index bb0f6366..cca0ce9c 100644
--- a/tests/test_inference_client.py
+++ b/tests/test_inference_client.py
@@ -200,6 +200,14 @@ class InferenceClientVCRTest(InferenceClientTest):
             " surpassed the Washington Monument to become the tallest man-made structure in the world.",
         )
 
+    def test_text_classification(self) -> None:
+        output = self.client.text_classification("I like you")
+        self.assertIsInstance(output, list)
+        self.assertEqual(len(output), 2)
+        for item in output:
+            self.assertIsInstance(item["score"], float)
+            self.assertIsInstance(item["label"], str)
+
     def test_text_generation(self) -> None:
         """Tested separately in `test_inference_text_generation.py`."""