Repository: huggingface/huggingface_hub
Pull request #1609 (2023-08-20 18:21:36)

Add question answering to inference client
**Add question-answering to HuggingFace🤗 Hub**

References https://github.com/huggingface/huggingface_hub/issues/1539

This is an ongoing list of model tasks to implement in the Hugging Face Hub inference client. Each task is planned to be its own PR. The task for this is [question-answering](https://huggingface.co/tasks/question-answering).

**Key Features**

- Modifies _client.py to call a question-answering model.
- Modifies testing and documentation to reflect changes.

HuggingFaceDocBuilderDev: The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/huggingface_hub/pr_1609). All of your documentation changes will be reflected on that endpoint.
codecov[bot]: ## [Codecov](https://app.codecov.io/gh/huggingface/huggingface_hub/pull/1609?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface) Report
Patch coverage: **`76.92%`** and project coverage change: **`-0.47%`** :warning:
> Comparison is base [(`e328730`)](https://app.codecov.io/gh/huggingface/huggingface_hub/commit/e328730821be171d42e39155d5ab08b52e160a9d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface) 82.26% compared to head [(`56b314c`)](https://app.codecov.io/gh/huggingface/huggingface_hub/pull/1609?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface) 81.79%.

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##             main    #1609      +/-   ##
==========================================
- Coverage   82.26%   81.79%   -0.47%     
==========================================
  Files          60       60              
  Lines        6777     6790      +13     
==========================================
- Hits         5575     5554      -21     
- Misses       1202     1236      +34     
```


| [Files Changed](https://app.codecov.io/gh/huggingface/huggingface_hub/pull/1609?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface) | Coverage Δ | |
|---|---|---|
| [...gingface\_hub/inference/\_generated/\_async\_client.py](https://app.codecov.io/gh/huggingface/huggingface_hub/pull/1609?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface#diff-c3JjL2h1Z2dpbmdmYWNlX2h1Yi9pbmZlcmVuY2UvX2dlbmVyYXRlZC9fYXN5bmNfY2xpZW50LnB5) | `58.37% <25.00%> (-0.62%)` | :arrow_down: |
| [src/huggingface\_hub/inference/\_client.py](https://app.codecov.io/gh/huggingface/huggingface_hub/pull/1609?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface#diff-c3JjL2h1Z2dpbmdmYWNlX2h1Yi9pbmZlcmVuY2UvX2NsaWVudC5weQ==) | `79.51% <100.00%> (+0.40%)` | :arrow_up: |
| [src/huggingface\_hub/inference/\_types.py](https://app.codecov.io/gh/huggingface/huggingface_hub/pull/1609?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface#diff-c3JjL2h1Z2dpbmdmYWNlX2h1Yi9pbmZlcmVuY2UvX3R5cGVzLnB5) | `96.15% <100.00%> (+0.91%)` | :arrow_up: |

... and [3 files with indirect coverage changes](https://app.codecov.io/gh/huggingface/huggingface_hub/pull/1609/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface)


</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/huggingface/huggingface_hub/pull/1609?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface).


Patch:
diff --git a/docs/source/en/guides/inference.md b/docs/source/en/guides/inference.md
index 10e5f0bf..7e766a44 100644
--- a/docs/source/en/guides/inference.md
+++ b/docs/source/en/guides/inference.md
@@ -135,7 +135,7 @@ has a simple API that supports the most common tasks. Here is a list of the curr
 | NLP | [Conversational](https://huggingface.co/tasks/conversational)                 | ✅ | [`~InferenceClient.conversational`] |
 | | [Feature Extraction](https://huggingface.co/tasks/feature-extraction)             | ✅ | [`~InferenceClient.feature_extraction`] |
 | | [Fill Mask](https://huggingface.co/tasks/fill-mask)                      |              |  |
-| | [Question Answering](https://huggingface.co/tasks/question-answering)             |              |  |
+| | [Question Answering](https://huggingface.co/tasks/question-answering)             | ✅ | [`~InferenceClient.question_answering`]
 | | [Sentence Similarity](https://huggingface.co/tasks/sentence-similarity)            | ✅ | [`~InferenceClient.sentence_similarity`] |
 | | [Summarization](https://huggingface.co/tasks/summarization)                  | ✅ | [`~InferenceClient.summarization`] |
 | | [Table Question Answering](https://huggingface.co/tasks/table-question-answering)       | |  |
diff --git a/src/huggingface_hub/inference/_client.py b/src/huggingface_hub/inference/_client.py
index f489dfd6..bae967b1 100644
--- a/src/huggingface_hub/inference/_client.py
+++ b/src/huggingface_hub/inference/_client.py
@@ -82,6 +82,7 @@ from huggingface_hub.inference._types import (
     ConversationalOutput,
     ImageSegmentationOutput,
     ObjectDetectionOutput,
+    QuestionAnsweringOutput,
     TokenClassificationOutput,
 )
 from huggingface_hub.utils import (
@@ -676,6 +677,47 @@ class InferenceClient:
             raise ValueError(f"Server output must be a list. Got {type(output)}: {str(output)[:200]}...")
         return output
 
+    def question_answering(
+        self, question: str, context: str, *, model: Optional[str] = None
+    ) -> QuestionAnsweringOutput:
+        """
+        Retrieve the answer to a question from a given text.
+
+        Args:
+            question (`str`):
+                Question to be answered.
+            context (`str`):
+                The context of the question.
+            model (`str`):
+                The model to use for the question answering task. Can be a model ID hosted on the Hugging Face Hub or a URL to
+                a deployed Inference Endpoint.
+
+        Returns:
+            `Dict`: a dictionary of question answering output containing the score, start index, end index, and answer.
+
+        Raises:
+            [`InferenceTimeoutError`]:
+                If the model is unavailable or the request times out.
+            `HTTPError`:
+                If the request fails with an HTTP error status code other than HTTP 503.
+
+        Example:
+        ```py
+        >>> from huggingface_hub import InferenceClient
+        >>> client = InferenceClient()
+        >>> client.question_answering(question="What's my name?", context="My name is Clara and I live in Berkeley.")
+        {'score': 0.9326562285423279, 'start': 11, 'end': 16, 'answer': 'Clara'}
+        ```
+        """
+
+        payload: Dict[str, Any] = {"question": question, "context": context}
+        response = self.post(
+            json=payload,
+            model=model,
+            task="question-answering",
+        )
+        return _bytes_to_dict(response)  # type: ignore
+
     def sentence_similarity(
         self, sentence: str, other_sentences: List[str], *, model: Optional[str] = None
     ) -> List[float]:
diff --git a/src/huggingface_hub/inference/_generated/_async_client.py b/src/huggingface_hub/inference/_generated/_async_client.py
index 59781d90..777dd26b 100644
--- a/src/huggingface_hub/inference/_generated/_async_client.py
+++ b/src/huggingface_hub/inference/_generated/_async_client.py
@@ -66,6 +66,7 @@ from huggingface_hub.inference._types import (
     ConversationalOutput,
     ImageSegmentationOutput,
     ObjectDetectionOutput,
+    QuestionAnsweringOutput,
     TokenClassificationOutput,
 )
 from huggingface_hub.utils import (
@@ -681,6 +682,48 @@ class AsyncInferenceClient:
             raise ValueError(f"Server output must be a list. Got {type(output)}: {str(output)[:200]}...")
         return output
 
+    async def question_answering(
+        self, question: str, context: str, *, model: Optional[str] = None
+    ) -> QuestionAnsweringOutput:
+        """
+        Retrieve the answer to a question from a given text.
+
+        Args:
+            question (`str`):
+                Question to be answered.
+            context (`str`):
+                The context of the question.
+            model (`str`):
+                The model to use for the question answering task. Can be a model ID hosted on the Hugging Face Hub or a URL to
+                a deployed Inference Endpoint.
+
+        Returns:
+            `Dict`: a dictionary of question answering output containing the score, start index, end index, and answer.
+
+        Raises:
+            [`InferenceTimeoutError`]:
+                If the model is unavailable or the request times out.
+            `aiohttp.ClientResponseError`:
+                If the request fails with an HTTP error status code other than HTTP 503.
+
+        Example:
+        ```py
+        # Must be run in an async context
+        >>> from huggingface_hub import AsyncInferenceClient
+        >>> client = AsyncInferenceClient()
+        >>> await client.question_answering(question="What's my name?", context="My name is Clara and I live in Berkeley.")
+        {'score': 0.9326562285423279, 'start': 11, 'end': 16, 'answer': 'Clara'}
+        ```
+        """
+
+        payload: Dict[str, Any] = {"question": question, "context": context}
+        response = await self.post(
+            json=payload,
+            model=model,
+            task="question-answering",
+        )
+        return _bytes_to_dict(response)  # type: ignore
+
     async def sentence_similarity(
         self, sentence: str, other_sentences: List[str], *, model: Optional[str] = None
     ) -> List[float]:
diff --git a/src/huggingface_hub/inference/_types.py b/src/huggingface_hub/inference/_types.py
index 9ea3d6aa..0fb9bb68 100644
--- a/src/huggingface_hub/inference/_types.py
+++ b/src/huggingface_hub/inference/_types.py
@@ -100,6 +100,26 @@ class ObjectDetectionOutput(TypedDict):
     score: float
 
 
+class QuestionAnsweringOutput(TypedDict):
+    """Dictionary containing information about a [`~InferenceClient.question_answering`] task.
+
+    Args:
+        score (`float`):
+            A float that represents how likely that the answer is correct.
+        start (`int`):
+            The index (string wise) of the start of the answer within context.
+        end (`int`):
+            The index (string wise) of the end of the answer within context.
+        answer (`str`):
+            A string that is the answer within the text.
+    """
+
+    score: float
+    start: int
+    end: int
+    answer: str
+
+
 class TokenClassificationOutput(TypedDict):
     """Dictionary containing the output of a [`~InferenceClient.token_classification`] task.
 


Test patch:
diff --git a/tests/cassettes/InferenceClientVCRTest.test_question_answering.yaml b/tests/cassettes/InferenceClientVCRTest.test_question_answering.yaml
new file mode 100644
index 00000000..5e6ee3c0
--- /dev/null
+++ b/tests/cassettes/InferenceClientVCRTest.test_question_answering.yaml
@@ -0,0 +1,48 @@
+interactions:
+- request:
+    body: '{"question": "What is the meaning of life?", "context": "42"}'
+    headers:
+      Accept:
+      - '*/*'
+      Accept-Encoding:
+      - gzip, deflate, br
+      Connection:
+      - keep-alive
+      Content-Length:
+      - '61'
+      Content-Type:
+      - application/json
+      X-Amzn-Trace-Id:
+      - e3071dca-bb69-47b9-b9c7-f0ce58a69927
+      user-agent:
+      - unknown/None; hf_hub/0.17.0.dev0; python/3.10.12
+    method: POST
+    uri: https://api-inference.huggingface.co/models/deepset/roberta-base-squad2
+  response:
+    body:
+      string: '{"score":1.4291124728060822e-08,"start":0,"end":2,"answer":"42"}'
+    headers:
+      Connection:
+      - keep-alive
+      Content-Length:
+      - '64'
+      Content-Type:
+      - application/json
+      Date:
+      - Sun, 20 Aug 2023 18:17:17 GMT
+      access-control-allow-credentials:
+      - 'true'
+      vary:
+      - Origin, Access-Control-Request-Method, Access-Control-Request-Headers
+      x-compute-time:
+      - '0.094'
+      x-compute-type:
+      - cache
+      x-request-id:
+      - vY1d3zhYMs71Bmhh1OI5N
+      x-sha:
+      - e09df911dd96d8b052d2665dfbb309e9398a9d70
+    status:
+      code: 200
+      message: OK
+version: 1
diff --git a/tests/test_inference_client.py b/tests/test_inference_client.py
index 32b5e02e..61dc5852 100644
--- a/tests/test_inference_client.py
+++ b/tests/test_inference_client.py
@@ -171,6 +171,16 @@ class InferenceClientVCRTest(InferenceClientTest):
             self.assertIn("xmax", item["box"])
             self.assertIn("ymax", item["box"])
 
+    def test_question_answering(self) -> None:
+        model = "deepset/roberta-base-squad2"
+        output = self.client.question_answering(question="What is the meaning of life?", context="42", model=model)
+        self.assertIsInstance(output, dict)
+        self.assertGreater(len(output), 0)
+        self.assertIsInstance(output["score"], float)
+        self.assertIsInstance(output["start"], int)
+        self.assertIsInstance(output["end"], int)
+        self.assertEqual(output["answer"], "42")
+
     def test_sentence_similarity(self) -> None:
         scores = self.client.sentence_similarity(
             "Machine learning is so easy.",