Repository: huggingface/huggingface_hub
Pull request #2094 (2024-03-07 13:10:19)

Implement `InferenceClient.chat_completion` + use new types for text-generation
Solve #2064.

Includes:
- types for text-generation + chat-completion outputs  (related to https://github.com/huggingface/huggingface.js/pull/537)
- support for the Chat completion in both `InferenceClient` and `AsyncInferenceClient`
- use text-generation types, including when stream=True
- make `minijinja` an optional dependency => only needed when models are not TGI-powered

### Workflow:

The workflow is a bit hacky depending on how the model is served. If the model/url is compatible with the Message API (which is the case for TGI-powered models), it's fairly straightforward. Otherwise we need to fetch the chat_completion config from the Hub, render the prompt and send a `text-generation` request. Rendering in done by [minijinja](https://github.com/mitsuhiko/minijinja) a lightweight rust-based template engine.

TL;DR: Works fine in most cases except for models running on transformers and served in Inference Endpoint. UX is quite bad for them (requests needs to be done by the user owning the inference endpoint). 

![chat_completion](https://github.com/huggingface/huggingface_hub/assets/11801849/6de03f79-d32f-4b5e-842a-04a268933e4c)

### TODO:
- [x] tests
- [x] documentation
- [x] handle TGI-served models
- [x] handle from TGI-based Inference Endpoint URL
- [x] handle from transformers-based Inference Endpoint URL

### Example (stream=True)

```py
from huggingface_hub import InferenceClient

client = InferenceClient("HuggingFaceH4/zephyr-7b-beta")

messages = [
    {"role": "system", "content": "You are a cook chef and always provide very good cooking advices."},
    {"role": "user", "content": "How can I cook spinach? List me 5 recipes with a very short description for each of them. I want diverse recipes from around the world. The spinach must be the main ingredient of the dish."},
]

# Print messages
for message in messages:
    print(message["role"].upper())
    print(message["content"].strip())
    print()

# Send request + print response
print("ASSISTANT")
response = ""
for item in client.chat_completion(messages=messages, max_tokens=10000, stream=True):
    if item.choices[0].finish_reason is None:
        delta_content = item.choices[0].delta.content
        response += delta_content
        print(delta_content, end="")
print("\n")
messages.append({"role": "assistant", "content": response})

# New user message
messages.append({"role": "user", "content": "Thank you! Could you give me the recipe for the third one with ingredient list and preparation steps?"})
print(messages[-1]["role"].upper())
print(messages[-1]["content"].strip())
print()

# Send new request + print response
print("ASSISTANT")
response = ""
for item in client.chat_completion(messages=messages, max_tokens=10000, stream=True):
    if item.choices[0].finish_reason is None:
        delta_content = item.choices[0].delta.content
        response += delta_content
        print(delta_content, end="")
print()
messages.append({"role": "assistant", "content": response})
```

### Output

```
SYSTEM
You are a cook chef and always provide very good cooking advices.

USER
How can I cook spinach? List me 5 recipes with a very short description for each of them. I want diverse recipes from around the world. The spinach must be the main ingredient of the dish.

ASSISTANT
1. Indian Palak Paneer: This vegetarian dish is a classic from North India. Sauté spinach with onions, garlic, and spices until wilted, then add crumbled paneer (Indian cheese) and simmer until heated through. Serve with rice or naan bread.

2. Greek Spanakorizo: This vegetarian dish is a traditional Greek rice dish made with spinach, rice, onions, garlic, and spices. It's often served as a side dish or as a vegetarian main course.

3. Italian Spinach and Ricotta Stuffed Shells: Cook jumbo pasta shells, then stuff them with a mixture of spinach, ricotta cheese, and spices. Bake in a tomato sauce until bubbly and golden brown.

4. Filipino Sinigang na Spinach: This sour soup is a staple in Filipino cuisine. Spinach is cooked with tomatoes, onions, garlic, and tamarind paste until tender, then seasoned with fish sauce and vinegar. Serve with rice.

5. Middle Eastern Spinach and Feta Stuffed Pita Bread: Cut pita bread in half and stuff with a mixture of spinach, feta cheese, onions, and spices. Bake in the oven until the bread is crispy and the filling is heated through. Serve with a side salad.

USER
Thank you! Could you give me the recipe for the third one with ingredient list and preparation steps?

ASSISTANT
Sure, here's the recipe for Spinach and Ricotta Stuffed Shells:

Ingredients:
- 12 jumbo pasta shells
- 1 (10-ounce) package frozen chopped spinach, thawed and drained
- 1 (15-ounce) container ricotta cheese
- 1/2 cup grated Parmesan cheese
- 1/2 cup grated mozzarella cheese
- 1/4 teaspoon salt
- 1/4 teaspoon black pepper
- 2 cloves garlic, minced
- 24-ounce jar marinara sauce
- 1/4 cup chopped fresh parsley

Preparation:

1. Preheat the oven to 375°F. Cook the pasta shells according to package directions. Drain and rinse with cold water to stop the cooking process.

2. In a large bowl, combine the spinach, ricotta cheese, Parmesan cheese, mozzarella cheese, salt, pepper, and garlic. Mix well.

3. Spoon the spinach mixture into the cooked pasta shells, filling each shell generously.

4. Pour the marinara sauce into the bottom of a 9x13-inch baking dish. Arrange the stuffed shells in the dish, placing them in a single layer.

5. Sprinkle chopped parsley over the top of the stuffed shells.

6. Bake for 25-30 minutes, or until the shells are heated through and the cheese is melted and bubbly.

7. Let the stuffed shells cool for a few minutes before serving. Enjoy!
```

HuggingFaceDocBuilderDev: The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/huggingface_hub/pr_2094). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
LysandreJik: cc @Rocketknight1 as well as it'd be great if this + the conversational implem in transformers were very similar
Wauplin: Thanks a lot for the thorough reviews @SBrandeis @LysandreJik! I think I've addressed all the comments so I'll soon merge it (waiting for the CI to complete). I'm glad to have this topic closed :smile: 

Patch:
diff --git a/docs/source/en/package_reference/inference_client.md b/docs/source/en/package_reference/inference_client.md
index 39b67c45..f25aaeeb 100644
--- a/docs/source/en/package_reference/inference_client.md
+++ b/docs/source/en/package_reference/inference_client.md
@@ -53,35 +53,6 @@ For most tasks, the return value has a built-in type (string, list, image...). H
 
 [[autodoc]] huggingface_hub.inference._common.ModelStatus
 
-### Text generation types
-
-[`~InferenceClient.text_generation`] task has a greater support than other tasks in `InferenceClient`. In
-particular, user inputs and server outputs are validated using [Pydantic](https://docs.pydantic.dev/latest/)
-if this package is installed. Therefore, we recommend installing it (`pip install pydantic`)
-for a better user experience.
-
-You can find below the dataclasses used to validate data and in particular [`~huggingface_hub.inference._text_generation.TextGenerationParameters`] (input),
-[`~huggingface_hub.inference._text_generation.TextGenerationResponse`] (output) and
-[`~huggingface_hub.inference._text_generation.TextGenerationStreamResponse`] (streaming output).
-
-[[autodoc]] huggingface_hub.inference._text_generation.TextGenerationParameters
-
-[[autodoc]] huggingface_hub.inference._text_generation.TextGenerationResponse
-
-[[autodoc]] huggingface_hub.inference._text_generation.TextGenerationStreamResponse
-
-[[autodoc]] huggingface_hub.inference._text_generation.InputToken
-
-[[autodoc]] huggingface_hub.inference._text_generation.Token
-
-[[autodoc]] huggingface_hub.inference._text_generation.FinishReason
-
-[[autodoc]] huggingface_hub.inference._text_generation.BestOfSequence
-
-[[autodoc]] huggingface_hub.inference._text_generation.Details
-
-[[autodoc]] huggingface_hub.inference._text_generation.StreamDetails
-
 ## InferenceAPI
 
 [`InferenceAPI`] is the legacy way to call the Inference API. The interface is more simplistic and requires knowing
diff --git a/docs/source/en/package_reference/inference_types.md b/docs/source/en/package_reference/inference_types.md
index 6d0160ae..0cb67764 100644
--- a/docs/source/en/package_reference/inference_types.md
+++ b/docs/source/en/package_reference/inference_types.md
@@ -49,6 +49,26 @@ This part of the lib is still under development and will be improved in future r
 
 
 
+## chat_completion
+
+[[autodoc]] huggingface_hub.ChatCompletionInput
+
+[[autodoc]] huggingface_hub.ChatCompletionInputMessage
+
+[[autodoc]] huggingface_hub.ChatCompletionOutput
+
+[[autodoc]] huggingface_hub.ChatCompletionOutputChoice
+
+[[autodoc]] huggingface_hub.ChatCompletionOutputChoiceMessage
+
+[[autodoc]] huggingface_hub.ChatCompletionStreamOutput
+
+[[autodoc]] huggingface_hub.ChatCompletionStreamOutputChoice
+
+[[autodoc]] huggingface_hub.ChatCompletionStreamOutputDelta
+
+
+
 ## depth_estimation
 
 [[autodoc]] huggingface_hub.DepthEstimationInput
@@ -203,19 +223,23 @@ This part of the lib is still under development and will be improved in future r
 
 ## text_generation
 
-[[autodoc]] huggingface_hub.PrefillToken
-
 [[autodoc]] huggingface_hub.TextGenerationInput
 
 [[autodoc]] huggingface_hub.TextGenerationOutput
 
 [[autodoc]] huggingface_hub.TextGenerationOutputDetails
 
+[[autodoc]] huggingface_hub.TextGenerationOutputSequenceDetails
+
+[[autodoc]] huggingface_hub.TextGenerationOutputToken
+
 [[autodoc]] huggingface_hub.TextGenerationParameters
 
-[[autodoc]] huggingface_hub.TextGenerationSequenceDetails
+[[autodoc]] huggingface_hub.TextGenerationPrefillToken
+
+[[autodoc]] huggingface_hub.TextGenerationStreamDetails
 
-[[autodoc]] huggingface_hub.Token
+[[autodoc]] huggingface_hub.TextGenerationStreamOutput
 
 
 
diff --git a/pyproject.toml b/pyproject.toml
index 9c4258a3..9537b280 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -1,9 +1,6 @@
 [tool.mypy]
 ignore_missing_imports = true
 no_implicit_optional = true
-plugins = [
-    "pydantic.mypy",
-]
 scripts_are_modules = true
 
 [tool.pytest.ini_options]
diff --git a/setup.py b/setup.py
index 61c7fd16..24b33446 100644
--- a/setup.py
+++ b/setup.py
@@ -14,28 +14,22 @@ def get_version() -> str:
 install_requires = [
     "filelock",
     "fsspec>=2023.5.0",
+    "packaging>=20.9",
+    "pyyaml>=5.1",
     "requests",
     "tqdm>=4.42.1",
-    "pyyaml>=5.1",
     "typing-extensions>=3.7.4.3",  # to be able to import TypeAlias
-    "packaging>=20.9",
 ]
 
 extras = {}
 
 extras["cli"] = [
-    "InquirerPy==0.3.4",
-    # Note: installs `prompt-toolkit` in the background
+    "InquirerPy==0.3.4",  # Note: installs `prompt-toolkit` in the background
 ]
 
 extras["inference"] = [
     "aiohttp",  # for AsyncInferenceClient
-    # On Python 3.8, Pydantic 2.x and tensorflow don't play well together
-    # Let's limit pydantic to 1.x for now. Since Tensorflow 2.14, Python3.8 is not supported anyway so impact should be
-    # limited. We still trigger some CIs on Python 3.8 so we need this workaround.
-    # NOTE: when relaxing constraint to support v3.x, make sure to adapt `src/huggingface_hub/inference/_text_generation.py`.
-    "pydantic>1.1,<3.0; python_version>'3.8'",
-    "pydantic>1.1,<2.0; python_version=='3.8'",
+    "minijinja>=1.0",  # for chat-completion if not TGI-served
 ]
 
 extras["torch"] = [
diff --git a/src/huggingface_hub/__init__.py b/src/huggingface_hub/__init__.py
index 960c7835..56378b05 100644
--- a/src/huggingface_hub/__init__.py
+++ b/src/huggingface_hub/__init__.py
@@ -270,6 +270,14 @@ _SUBMOD_ATTRS = {
         "AutomaticSpeechRecognitionOutput",
         "AutomaticSpeechRecognitionOutputChunk",
         "AutomaticSpeechRecognitionParameters",
+        "ChatCompletionInput",
+        "ChatCompletionInputMessage",
+        "ChatCompletionOutput",
+        "ChatCompletionOutputChoice",
+        "ChatCompletionOutputChoiceMessage",
+        "ChatCompletionStreamOutput",
+        "ChatCompletionStreamOutputChoice",
+        "ChatCompletionStreamOutputDelta",
         "DepthEstimationInput",
         "DepthEstimationOutput",
         "DocumentQuestionAnsweringInput",
@@ -298,7 +306,6 @@ _SUBMOD_ATTRS = {
         "ObjectDetectionInput",
         "ObjectDetectionOutputElement",
         "ObjectDetectionParameters",
-        "PrefillToken",
         "QuestionAnsweringInput",
         "QuestionAnsweringInputData",
         "QuestionAnsweringOutputElement",
@@ -320,8 +327,12 @@ _SUBMOD_ATTRS = {
         "TextGenerationInput",
         "TextGenerationOutput",
         "TextGenerationOutputDetails",
+        "TextGenerationOutputSequenceDetails",
+        "TextGenerationOutputToken",
         "TextGenerationParameters",
-        "TextGenerationSequenceDetails",
+        "TextGenerationPrefillToken",
+        "TextGenerationStreamDetails",
+        "TextGenerationStreamOutput",
         "TextToAudioGenerationParameters",
         "TextToAudioInput",
         "TextToAudioOutput",
@@ -330,7 +341,6 @@ _SUBMOD_ATTRS = {
         "TextToImageOutput",
         "TextToImageParameters",
         "TextToImageTargetSize",
-        "Token",
         "TokenClassificationInput",
         "TokenClassificationOutputElement",
         "TokenClassificationParameters",
@@ -722,6 +732,14 @@ if TYPE_CHECKING:  # pragma: no cover
         AutomaticSpeechRecognitionOutput,  # noqa: F401
         AutomaticSpeechRecognitionOutputChunk,  # noqa: F401
         AutomaticSpeechRecognitionParameters,  # noqa: F401
+        ChatCompletionInput,  # noqa: F401
+        ChatCompletionInputMessage,  # noqa: F401
+        ChatCompletionOutput,  # noqa: F401
+        ChatCompletionOutputChoice,  # noqa: F401
+        ChatCompletionOutputChoiceMessage,  # noqa: F401
+        ChatCompletionStreamOutput,  # noqa: F401
+        ChatCompletionStreamOutputChoice,  # noqa: F401
+        ChatCompletionStreamOutputDelta,  # noqa: F401
         DepthEstimationInput,  # noqa: F401
         DepthEstimationOutput,  # noqa: F401
         DocumentQuestionAnsweringInput,  # noqa: F401
@@ -750,7 +768,6 @@ if TYPE_CHECKING:  # pragma: no cover
         ObjectDetectionInput,  # noqa: F401
         ObjectDetectionOutputElement,  # noqa: F401
         ObjectDetectionParameters,  # noqa: F401
-        PrefillToken,  # noqa: F401
         QuestionAnsweringInput,  # noqa: F401
         QuestionAnsweringInputData,  # noqa: F401
         QuestionAnsweringOutputElement,  # noqa: F401
@@ -772,8 +789,12 @@ if TYPE_CHECKING:  # pragma: no cover
         TextGenerationInput,  # noqa: F401
         TextGenerationOutput,  # noqa: F401
         TextGenerationOutputDetails,  # noqa: F401
+        TextGenerationOutputSequenceDetails,  # noqa: F401
+        TextGenerationOutputToken,  # noqa: F401
         TextGenerationParameters,  # noqa: F401
-        TextGenerationSequenceDetails,  # noqa: F401
+        TextGenerationPrefillToken,  # noqa: F401
+        TextGenerationStreamDetails,  # noqa: F401
+        TextGenerationStreamOutput,  # noqa: F401
         TextToAudioGenerationParameters,  # noqa: F401
         TextToAudioInput,  # noqa: F401
         TextToAudioOutput,  # noqa: F401
@@ -782,7 +803,6 @@ if TYPE_CHECKING:  # pragma: no cover
         TextToImageOutput,  # noqa: F401
         TextToImageParameters,  # noqa: F401
         TextToImageTargetSize,  # noqa: F401
-        Token,  # noqa: F401
         TokenClassificationInput,  # noqa: F401
         TokenClassificationOutputElement,  # noqa: F401
         TokenClassificationParameters,  # noqa: F401
diff --git a/src/huggingface_hub/_inference_endpoints.py b/src/huggingface_hub/_inference_endpoints.py
index 90b9d324..8b81a796 100644
--- a/src/huggingface_hub/_inference_endpoints.py
+++ b/src/huggingface_hub/_inference_endpoints.py
@@ -2,7 +2,7 @@ import time
 from dataclasses import dataclass, field
 from datetime import datetime
 from enum import Enum
-from typing import TYPE_CHECKING, Dict, Optional
+from typing import TYPE_CHECKING, Dict, Optional, Union
 
 from .inference._client import InferenceClient
 from .inference._generated._async_client import AsyncInferenceClient
@@ -71,8 +71,9 @@ class InferenceEndpoint:
             The type of the Inference Endpoint (public, protected, private).
         raw (`Dict`):
             The raw dictionary data returned from the API.
-        token (`str`, *optional*):
-            Authentication token for the Inference Endpoint, if set when requesting the API.
+        token (`str` or `bool`, *optional*):
+            Authentication token for the Inference Endpoint, if set when requesting the API. Will default to the
+            locally saved token if not provided. Pass `token=False` if you don't want to send your token to the server.
 
     Example:
         ```python
@@ -120,12 +121,12 @@ class InferenceEndpoint:
     raw: Dict = field(repr=False)
 
     # Internal fields
-    _token: Optional[str] = field(repr=False, compare=False)
+    _token: Union[str, bool, None] = field(repr=False, compare=False)
     _api: "HfApi" = field(repr=False, compare=False)
 
     @classmethod
     def from_raw(
-        cls, raw: Dict, namespace: str, token: Optional[str] = None, api: Optional["HfApi"] = None
+        cls, raw: Dict, namespace: str, token: Union[str, bool, None] = None, api: Optional["HfApi"] = None
     ) -> "InferenceEndpoint":
         """Initialize object from raw dictionary."""
         if api is None:
@@ -230,7 +231,7 @@ class InferenceEndpoint:
         Returns:
             [`InferenceEndpoint`]: the same Inference Endpoint, mutated in place with the latest data.
         """
-        obj = self._api.get_inference_endpoint(name=self.name, namespace=self.namespace, token=self._token)
+        obj = self._api.get_inference_endpoint(name=self.name, namespace=self.namespace, token=self._token)  # type: ignore [arg-type]
         self.raw = obj.raw
         self._populate_from_raw()
         return self
@@ -295,7 +296,7 @@ class InferenceEndpoint:
             framework=framework,
             revision=revision,
             task=task,
-            token=self._token,
+            token=self._token,  # type: ignore [arg-type]
         )
 
         # Mutate current object
@@ -316,7 +317,7 @@ class InferenceEndpoint:
         Returns:
             [`InferenceEndpoint`]: the same Inference Endpoint, mutated in place with the latest data.
         """
-        obj = self._api.pause_inference_endpoint(name=self.name, namespace=self.namespace, token=self._token)
+        obj = self._api.pause_inference_endpoint(name=self.name, namespace=self.namespace, token=self._token)  # type: ignore [arg-type]
         self.raw = obj.raw
         self._populate_from_raw()
         return self
@@ -330,7 +331,7 @@ class InferenceEndpoint:
         Returns:
             [`InferenceEndpoint`]: the same Inference Endpoint, mutated in place with the latest data.
         """
-        obj = self._api.resume_inference_endpoint(name=self.name, namespace=self.namespace, token=self._token)
+        obj = self._api.resume_inference_endpoint(name=self.name, namespace=self.namespace, token=self._token)  # type: ignore [arg-type]
         self.raw = obj.raw
         self._populate_from_raw()
         return self
@@ -348,7 +349,7 @@ class InferenceEndpoint:
         Returns:
             [`InferenceEndpoint`]: the same Inference Endpoint, mutated in place with the latest data.
         """
-        obj = self._api.scale_to_zero_inference_endpoint(name=self.name, namespace=self.namespace, token=self._token)
+        obj = self._api.scale_to_zero_inference_endpoint(name=self.name, namespace=self.namespace, token=self._token)  # type: ignore [arg-type]
         self.raw = obj.raw
         self._populate_from_raw()
         return self
@@ -361,7 +362,7 @@ class InferenceEndpoint:
 
         This is an alias for [`HfApi.delete_inference_endpoint`].
         """
-        self._api.delete_inference_endpoint(name=self.name, namespace=self.namespace, token=self._token)
+        self._api.delete_inference_endpoint(name=self.name, namespace=self.namespace, token=self._token)  # type: ignore [arg-type]
 
     def _populate_from_raw(self) -> None:
         """Populate fields from raw dictionary.
diff --git a/src/huggingface_hub/hf_api.py b/src/huggingface_hub/hf_api.py
index 60417a4c..5b00c489 100644
--- a/src/huggingface_hub/hf_api.py
+++ b/src/huggingface_hub/hf_api.py
@@ -1242,7 +1242,7 @@ class HfApi:
     def __init__(
         self,
         endpoint: Optional[str] = None,
-        token: Optional[str] = None,
+        token: Union[str, bool, None] = None,
         library_name: Optional[str] = None,
         library_version: Optional[str] = None,
         user_agent: Union[Dict, str, None] = None,
@@ -1256,9 +1256,9 @@ class HfApi:
         directly at the root of `huggingface_hub`.
 
         Args:
-            token (`str`, *optional*):
-                Hugging Face token. Will default to the locally saved token if
-                not provided.
+            token (`str` or `bool`, *optional*):
+                Hugging Face token. Will default to the locally saved token if not provided.
+                Pass `token=False` if you don't want to send your token to the server.
             library_name (`str`, *optional*):
                 The name of the library that is making the HTTP request. Will be added to
                 the user-agent header. Example: `"transformers"`.
@@ -2527,7 +2527,7 @@ class HfApi:
         *,
         repo_type: Optional[str] = None,
         revision: Optional[str] = None,
-        token: Optional[str] = None,
+        token: Union[str, bool, None] = None,
     ) -> bool:
         """
         Checks if a file exists in a repository on the Hugging Face Hub.
diff --git a/src/huggingface_hub/inference/_client.py b/src/huggingface_hub/inference/_client.py
index 4fc2963a..6a07ef65 100644
--- a/src/huggingface_hub/inference/_client.py
+++ b/src/huggingface_hub/inference/_client.py
@@ -36,7 +36,6 @@ import base64
 import logging
 import time
 import warnings
-from dataclasses import asdict
 from typing import (
     TYPE_CHECKING,
     Any,
@@ -65,15 +64,24 @@ from huggingface_hub.inference._common import (
     _bytes_to_list,
     _fetch_recommended_models,
     _import_numpy,
+    _is_chat_completion_server,
     _is_tgi_server,
     _open_as_binary,
+    _set_as_non_chat_completion_server,
     _set_as_non_tgi,
+    _stream_chat_completion_response_from_bytes,
+    _stream_chat_completion_response_from_text_generation,
     _stream_text_generation_response,
+    raise_text_generation_error,
 )
 from huggingface_hub.inference._generated.types import (
     AudioClassificationOutputElement,
     AudioToAudioOutputElement,
     AutomaticSpeechRecognitionOutput,
+    ChatCompletionOutput,
+    ChatCompletionOutputChoice,
+    ChatCompletionOutputChoiceMessage,
+    ChatCompletionStreamOutput,
     DocumentQuestionAnsweringOutputElement,
     FillMaskOutputElement,
     ImageClassificationOutputElement,
@@ -84,19 +92,15 @@ from huggingface_hub.inference._generated.types import (
     SummarizationOutput,
     TableQuestionAnsweringOutputElement,
     TextClassificationOutputElement,
+    TextGenerationOutput,
+    TextGenerationStreamOutput,
     TokenClassificationOutputElement,
     TranslationOutput,
     VisualQuestionAnsweringOutputElement,
     ZeroShotClassificationOutputElement,
     ZeroShotImageClassificationOutputElement,
 )
-from huggingface_hub.inference._text_generation import (
-    TextGenerationParameters,
-    TextGenerationRequest,
-    TextGenerationResponse,
-    TextGenerationStreamResponse,
-    raise_text_generation_error,
-)
+from huggingface_hub.inference._templating import render_chat_prompt
 from huggingface_hub.inference._types import (
     ConversationalOutput,  # soon to be removed
 )
@@ -127,9 +131,9 @@ class InferenceClient:
             The model to run inference with. Can be a model id hosted on the Hugging Face Hub, e.g. `bigcode/starcoder`
             or a URL to a deployed Inference Endpoint. Defaults to None, in which case a recommended model is
             automatically selected for the task.
-        token (`str`, *optional*):
-            Hugging Face token. Will default to the locally saved token. Pass `token=False` if you don't want to send
-            your token to the server.
+        token (`str` or `bool`, *optional*):
+            Hugging Face token. Will default to the locally saved token if not provided.
+            Pass `token=False` if you don't want to send your token to the server.
         timeout (`float`, `optional`):
             The maximum number of seconds to wait for a response from the server. Loading a new model in Inference
             API can take up to several minutes. Defaults to None, meaning it will loop until the server is available.
@@ -149,6 +153,7 @@ class InferenceClient:
         cookies: Optional[Dict[str, str]] = None,
     ) -> None:
         self.model: Optional[str] = model
+        self.token: Union[str, bool, None] = token
         self.headers = CaseInsensitiveDict(build_hf_headers(token=token))  # contains 'authorization' + 'user-agent'
         if headers is not None:
             self.headers.update(headers)
@@ -167,11 +172,10 @@ class InferenceClient:
         model: Optional[str] = None,
         task: Optional[str] = None,
         stream: Literal[False] = ...,
-    ) -> bytes:
-        pass
+    ) -> bytes: ...
 
     @overload
-    def post(
+    def post(  # type: ignore[misc]
         self,
         *,
         json: Optional[Union[str, Dict, List]] = None,
@@ -179,8 +183,18 @@ class InferenceClient:
         model: Optional[str] = None,
         task: Optional[str] = None,
         stream: Literal[True] = ...,
-    ) -> Iterable[bytes]:
-        pass
+    ) -> Iterable[bytes]: ...
+
+    @overload
+    def post(
+        self,
+        *,
+        json: Optional[Union[str, Dict, List]] = None,
+        data: Optional[ContentT] = None,
+        model: Optional[str] = None,
+        task: Optional[str] = None,
+        stream: bool = False,
+    ) -> Union[bytes, Iterable[bytes]]: ...
 
     def post(
         self,
@@ -395,6 +409,259 @@ class InferenceClient:
         response = self.post(data=audio, model=model, task="automatic-speech-recognition")
         return AutomaticSpeechRecognitionOutput.parse_obj_as_instance(response)
 
+    @overload
+    def chat_completion(  # type: ignore
+        self,
+        messages: List[Dict[str, str]],
+        *,
+        model: Optional[str] = None,
+        stream: Literal[False] = False,
+        max_tokens: int = 20,
+        seed: Optional[int] = None,
+        stop: Optional[Union[List[str], str]] = None,
+        temperature: float = 1.0,
+        top_p: Optional[float] = None,
+    ) -> ChatCompletionOutput: ...
+
+    @overload
+    def chat_completion(  # type: ignore
+        self,
+        messages: List[Dict[str, str]],
+        *,
+        model: Optional[str] = None,
+        stream: Literal[True] = True,
+        max_tokens: int = 20,
+        seed: Optional[int] = None,
+        stop: Optional[Union[List[str], str]] = None,
+        temperature: float = 1.0,
+        top_p: Optional[float] = None,
+    ) -> Iterable[ChatCompletionStreamOutput]: ...
+
+    @overload
+    def chat_completion(
+        self,
+        messages: List[Dict[str, str]],
+        *,
+        model: Optional[str] = None,
+        stream: bool = False,
+        max_tokens: int = 20,
+        seed: Optional[int] = None,
+        stop: Optional[Union[List[str], str]] = None,
+        temperature: float = 1.0,
+        top_p: Optional[float] = None,
+    ) -> Union[ChatCompletionOutput, Iterable[ChatCompletionStreamOutput]]: ...
+
+    def chat_completion(
+        self,
+        messages: List[Dict[str, str]],
+        *,
+        model: Optional[str] = None,
+        stream: bool = False,
+        max_tokens: int = 20,
+        seed: Optional[int] = None,
+        stop: Optional[Union[List[str], str]] = None,
+        temperature: float = 1.0,
+        top_p: Optional[float] = None,
+    ) -> Union[ChatCompletionOutput, Iterable[ChatCompletionStreamOutput]]:
+        """
+        A method for completing conversations using a specified language model.
+
+        <Tip>
+
+        If the model is served by a server supporting chat-completion, the method will directly call the server's
+        `/v1/chat/completions` endpoint. If the server does not support chat-completion, the method will render the
+        chat template client-side based on the information fetched from the Hub API. In this case, you will need to
+        have `minijinja` template engine installed. Run `pip install "huggingface_hub[inference]"` or `pip install minijinja`
+        to install it.
+
+        </Tip>
+
+        Args:
+            messages (List[Union[`SystemMessage`, `UserMessage`, `AssistantMessage`]]):
+                Conversation history consisting of roles and content pairs.
+            model (`str`, *optional*):
+                The model to use for chat-completion. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed
+                Inference Endpoint. If not provided, the default recommended model for chat-based text-generation will be used.
+                See https://huggingface.co/tasks/text-generation for more details.
+            frequency_penalty (`float`, optional):
+                Penalizes new tokens based on their existing frequency
+                in the text so far. Range: [-2.0, 2.0]. Defaults to 0.0.
+            max_tokens (`int`, optional):
+                Maximum number of tokens allowed in the response. Defaults to 20.
+            seed (Optional[`int`], optional):
+                Seed for reproducible control flow. Defaults to None.
+            stop (Optional[`str`], optional):
+                Up to four strings which trigger the end of the response.
+                Defaults to None.
+            stream (`bool`, optional):
+                Enable realtime streaming of responses. Defaults to False.
+            temperature (`float`, optional):
+                Controls randomness of the generations. Lower values ensure
+                less random completions. Range: [0, 2]. Defaults to 1.0.
+            top_p (`float`, optional):
+                Fraction of the most likely next words to sample from.
+                Must be between 0 and 1. Defaults to 1.0.
+
+        Returns:
+            `Union[ChatCompletionOutput, Iterable[ChatCompletionStreamOutput]]`:
+            Generated text returned from the server:
+            - if `stream=False`, the generated text is returned as a [`ChatCompletionOutput`] (default).
+            - if `stream=True`, the generated text is returned token by token as a sequence of [`ChatCompletionStreamOutput`].
+
+        Raises:
+            [`InferenceTimeoutError`]:
+                If the model is unavailable or the request times out.
+            `HTTPError`:
+                If the request fails with an HTTP error status code other than HTTP 503.
+
+        Example:
+        ```py
+        >>> from huggingface_hub import InferenceClient
+        >>> messages = [{"role": "user", "content": "What is the capital of France?"}]
+        >>> client = InferenceClient("HuggingFaceH4/zephyr-7b-beta")
+        >>> client.chat_completion(messages, max_tokens=100)
+        ChatCompletionOutput(
+            choices=[
+                ChatCompletionOutputChoice(
+                    finish_reason='eos_token',
+                    index=0,
+                    message=ChatCompletionOutputChoiceMessage(
+                        content='The capital of France is Paris. The official name of the city is "Ville de Paris" (City of Paris) and the name of the country\'s governing body, which is located in Paris, is "La République française" (The French Republic). \nI hope that helps! Let me know if you need any further information.'
+                    )
+                )
+            ],
+            created=1710498360
+        )
+
+        >>> for token in client.chat_completion(messages, max_tokens=10, stream=True):
+        ...     print(token)
+        ChatCompletionStreamOutput(choices=[ChatCompletionStreamOutputChoice(delta=ChatCompletionStreamOutputDelta(content='The', role='assistant'), index=0, finish_reason=None)], created=1710498504)
+        ChatCompletionStreamOutput(choices=[ChatCompletionStreamOutputChoice(delta=ChatCompletionStreamOutputDelta(content=' capital', role='assistant'), index=0, finish_reason=None)], created=1710498504)
+        (...)
+        ChatCompletionStreamOutput(choices=[ChatCompletionStreamOutputChoice(delta=ChatCompletionStreamOutputDelta(content=' may', role='assistant'), index=0, finish_reason=None)], created=1710498504)
+        ChatCompletionStreamOutput(choices=[ChatCompletionStreamOutputChoice(delta=ChatCompletionStreamOutputDelta(content=None, role=None), index=0, finish_reason='length')], created=1710498504)
+        ```
+        """
+        # determine model
+        model = model or self.model or self.get_recommended_model("text-generation")
+
+        if _is_chat_completion_server(model):
+            # First, let's consider the server has a `/v1/chat/completions` endpoint.
+            # If that's the case, we don't have to render the chat template client-side.
+            model_url = self._resolve_url(model) + "/v1/chat/completions"
+
+            try:
+                data = self.post(
+                    model=model_url,
+                    json=dict(
+                        model="tgi",  # random string
+                        messages=messages,
+                        max_tokens=max_tokens,
+                        seed=seed,
+                        stop=stop,
+                        temperature=temperature,
+                        top_p=top_p,
+                        stream=stream,
+                    ),
+                    stream=stream,
+                )
+            except HTTPError:
+                # Let's consider the server is not a chat completion server.
+                # Then we call again `chat_completion` which will render the chat template client side.
+                # (can be HTTP 500, HTTP 400, HTTP 404 depending on the server)
+                _set_as_non_chat_completion_server(model)
+                return self.chat_completion(
+                    messages=messages,
+                    model=model,
+                    stream=stream,
+                    max_tokens=max_tokens,
+                    seed=seed,
+                    stop=stop,
+                    temperature=temperature,
+                    top_p=top_p,
+                )
+
+            if stream:
+                return _stream_chat_completion_response_from_bytes(data)  # type: ignore[arg-type]
+
+            return ChatCompletionOutput.parse_obj_as_instance(data)  # type: ignore[arg-type]
+
+        # At this point, we know the server is not a chat completion server.
+        # We need to render the chat template client side based on the information we can fetch from
+        # the Hub API.
+
+        model_id = None
+        if model.startswith(("http://", "https://")):
+            # If URL, we need to know which model is served. This is not always possible.
+            # A workaround is to list the user Inference Endpoints and check if one of them correspond to the model URL.
+            # If not, we raise an error.
+            # TODO: fix when we have a proper API for this (at least for Inference Endpoints)
+            # TODO: what if Sagemaker URL?
+            # TODO: what if Azure URL?
+            from ..hf_api import HfApi
+
+            for endpoint in HfApi(token=self.token).list_inference_endpoints():
+                if endpoint.url == model:
+                    model_id = endpoint.repository
+                    break
+        else:
+            model_id = model
+
+        if model_id is None:
+            # If we don't have the model ID, we can't fetch the chat template.
+            # We raise an error.
+            raise ValueError(
+                "Request can't be processed as the model ID can't be inferred from model URL. "
+                "This is needed to fetch the chat template from the Hub since the model is not "
+                "served with a Chat-completion API."
+            )
+
+        # fetch chat template + tokens
+        prompt = render_chat_prompt(model_id=model_id, token=self.token, messages=messages)
+
+        # generate response
+        stop_sequences = [stop] if isinstance(stop, str) else stop
+        text_generation_output = self.text_generation(
+            prompt=prompt,
+            details=True,
+            stream=stream,
+            model=model,
+            max_new_tokens=max_tokens,
+            seed=seed,
+            stop_sequences=stop_sequences,
+            temperature=temperature,
+            top_p=top_p,
+        )
+
+        created = int(time.time())
+
+        if stream:
+            return _stream_chat_completion_response_from_text_generation(text_generation_output)  # type: ignore [arg-type]
+
+        if isinstance(text_generation_output, TextGenerationOutput):
+            # General use case => format ChatCompletionOutput from text generation details
+            content: str = text_generation_output.generated_text
+            finish_reason: str = text_generation_output.details.finish_reason  # type: ignore[union-attr]
+        else:
+            # Corner case: if server doesn't support details (e.g. if not a TGI server), we only receive an output string.
+            # In such a case, `finish_reason` is set to `"unk"`.
+            content = text_generation_output  # type: ignore[assignment]
+            finish_reason = "unk"
+
+        return ChatCompletionOutput(
+            created=created,
+            choices=[
+                ChatCompletionOutputChoice(
+                    finish_reason=finish_reason,  # type: ignore
+                    index=0,
+                    message=ChatCompletionOutputChoiceMessage(
+                        content=content,
+                        role="assistant",
+                    ),
+                )
+            ],
+        )
+
     def conversational(
         self,
         text: str,
@@ -407,6 +674,13 @@ class InferenceClient:
         """
         Generate conversational responses based on the given input text (i.e. chat with the API).
 
+        <Tip warning={true}>
+
+        [`InferenceClient.conversational`] API is deprecated and will be removed in a future release. Please use
+        [`InferenceClient.chat_completion`] instead.
+
+        </Tip>
+
         Args:
             text (`str`):
                 The last input from the user in the conversation.
@@ -446,6 +720,11 @@ class InferenceClient:
         ... )
         ```
         """
+        warnings.warn(
+            "'InferenceClient.conversational' is deprecated and will be removed starting from huggingface_hub>=0.25. "
+            "Please use the more appropriate 'InferenceClient.chat_completion' API instead.",
+            FutureWarning,
+        )
         payload: Dict[str, Any] = {"inputs": {"text": text}}
         if generated_responses is not None:
             payload["inputs"]["generated_responses"] = generated_responses
@@ -1230,7 +1509,7 @@ class InferenceClient:
         truncate: Optional[int] = None,
         typical_p: Optional[float] = None,
         watermark: bool = False,
-    ) -> TextGenerationResponse: ...
+    ) -> TextGenerationOutput: ...
 
     @overload
     def text_generation(  # type: ignore
@@ -1256,7 +1535,7 @@ class InferenceClient:
     ) -> Iterable[str]: ...
 
     @overload
-    def text_generation(
+    def text_generation(  # type: ignore
         self,
         prompt: str,
         *,
@@ -1276,7 +1555,30 @@ class InferenceClient:
         truncate: Optional[int] = None,
         typical_p: Optional[float] = None,
         watermark: bool = False,
-    ) -> Iterable[TextGenerationStreamResponse]: ...
+    ) -> Iterable[TextGenerationStreamOutput]: ...
+
+    @overload
+    def text_generation(
+        self,
+        prompt: str,
+        *,
+        details: Literal[True] = ...,
+        stream: bool = ...,
+        model: Optional[str] = None,
+        do_sample: bool = False,
+        max_new_tokens: int = 20,
+        best_of: Optional[int] = None,
+        repetition_penalty: Optional[float] = None,
+        return_full_text: bool = False,
+        seed: Optional[int] = None,
+        stop_sequences: Optional[List[str]] = None,
+        temperature: Optional[float] = None,
+        top_k: Optional[int] = None,
+        top_p: Optional[float] = None,
+        truncate: Optional[int] = None,
+        typical_p: Optional[float] = None,
+        watermark: bool = False,
+    ) -> Union[TextGenerationOutput, Iterable[TextGenerationStreamOutput]]: ...
 
     def text_generation(
         self,
@@ -1299,13 +1601,10 @@ class InferenceClient:
         typical_p: Optional[float] = None,
         watermark: bool = False,
         decoder_input_details: bool = False,
-    ) -> Union[str, TextGenerationResponse, Iterable[str], Iterable[TextGenerationStreamResponse]]:
+    ) -> Union[str, TextGenerationOutput, Iterable[str], Iterable[TextGenerationStreamOutput]]:
         """
         Given a prompt, generate the following text.
 
-        It is recommended to have Pydantic installed in order to get inputs validated. This is preferable as it allow
-        early failures.
-
         API endpoint is supposed to run with the `text-generation-inference` backend (TGI). This backend is the
         go-to solution to run large language models at scale. However, for some smaller models (e.g. "gpt2") the
         default `transformers` + `api-inference` solution is still in use. Both approaches have very similar APIs, but
@@ -1363,12 +1662,12 @@ class InferenceClient:
                 into account. Defaults to `False`.
 
         Returns:
-            `Union[str, TextGenerationResponse, Iterable[str], Iterable[TextGenerationStreamResponse]]`:
+            `Union[str, TextGenerationOutput, Iterable[str], Iterable[TextGenerationStreamOutput]]`:
             Generated text returned from the server:
             - if `stream=False` and `details=False`, the generated text is returned as a `str` (default)
             - if `stream=True` and `details=False`, the generated text is returned token by token as a `Iterable[str]`
-            - if `stream=False` and `details=True`, the generated text is returned with more details as a [`~huggingface_hub.inference._text_generation.TextGenerationResponse`]
-            - if `details=True` and `stream=True`, the generated text is returned token by token as a iterable of [`~huggingface_hub.inference._text_generation.TextGenerationStreamResponse`]
+            - if `stream=False` and `details=True`, the generated text is returned with more details as a [`~huggingface_hub.TextGenerationOutput`]
+            - if `details=True` and `stream=True`, the generated text is returned token by token as a iterable of [`~huggingface_hub.TextGenerationStreamOutput`]
 
         Raises:
             `ValidationError`:
@@ -1405,23 +1704,23 @@ class InferenceClient:
 
         # Case 3: get more details about the generation process.
         >>> client.text_generation("The huggingface_hub library is ", max_new_tokens=12, details=True)
-        TextGenerationResponse(
+        TextGenerationOutput(
             generated_text='100% open source and built to be easy to use.',
-            details=Details(
-                finish_reason=<FinishReason.Length: 'length'>,
+            details=TextGenerationDetails(
+                finish_reason='length',
                 generated_tokens=12,
                 seed=None,
                 prefill=[
-                    InputToken(id=487, text='The', logprob=None),
-                    InputToken(id=53789, text=' hugging', logprob=-13.171875),
+                    TextGenerationPrefillToken(id=487, text='The', logprob=None),
+                    TextGenerationPrefillToken(id=53789, text=' hugging', logprob=-13.171875),
                     (...)
-                    InputToken(id=204, text=' ', logprob=-7.0390625)
+                    TextGenerationPrefillToken(id=204, text=' ', logprob=-7.0390625)
                 ],
                 tokens=[
-                    Token(id=1425, text='100', logprob=-1.0175781, special=False),
-                    Token(id=16, text='%', logprob=-0.0463562, special=False),
+                    TokenElement(id=1425, text='100', logprob=-1.0175781, special=False),
+                    TokenElement(id=16, text='%', logprob=-0.0463562, special=False),
                     (...)
-                    Token(id=25, text='.', logprob=-0.5703125, special=False)
+                    TokenElement(id=25, text='.', logprob=-0.5703125, special=False)
                 ],
                 best_of_sequences=None
             )
@@ -1432,30 +1731,27 @@ class InferenceClient:
         >>> for details in client.text_generation("The huggingface_hub library is ", max_new_tokens=12, details=True, stream=True):
         ...     print(details)
         ...
-        TextGenerationStreamResponse(token=Token(id=1425, text='100', logprob=-1.0175781, special=False), generated_text=None, details=None)
-        TextGenerationStreamResponse(token=Token(id=16, text='%', logprob=-0.0463562, special=False), generated_text=None, details=None)
-        TextGenerationStreamResponse(token=Token(id=1314, text=' open', logprob=-1.3359375, special=False), generated_text=None, details=None)
-        TextGenerationStreamResponse(token=Token(id=3178, text=' source', logprob=-0.28100586, special=False), generated_text=None, details=None)
-        TextGenerationStreamResponse(token=Token(id=273, text=' and', logprob=-0.5961914, special=False), generated_text=None, details=None)
-        TextGenerationStreamResponse(token=Token(id=3426, text=' built', logprob=-1.9423828, special=False), generated_text=None, details=None)
-        TextGenerationStreamResponse(token=Token(id=271, text=' to', logprob=-1.4121094, special=False), generated_text=None, details=None)
-        TextGenerationStreamResponse(token=Token(id=314, text=' be', logprob=-1.5224609, special=False), generated_text=None, details=None)
-        TextGenerationStreamResponse(token=Token(id=1833, text=' easy', logprob=-2.1132812, special=False), generated_text=None, details=None)
-        TextGenerationStreamResponse(token=Token(id=271, text=' to', logprob=-0.08520508, special=False), generated_text=None, details=None)
-        TextGenerationStreamResponse(token=Token(id=745, text=' use', logprob=-0.39453125, special=False), generated_text=None, details=None)
-        TextGenerationStreamResponse(token=Token(
+        TextGenerationStreamOutput(token=TokenElement(id=1425, text='100', logprob=-1.0175781, special=False), generated_text=None, details=None)
+        TextGenerationStreamOutput(token=TokenElement(id=16, text='%', logprob=-0.0463562, special=False), generated_text=None, details=None)
+        TextGenerationStreamOutput(token=TokenElement(id=1314, text=' open', logprob=-1.3359375, special=False), generated_text=None, details=None)
+        TextGenerationStreamOutput(token=TokenElement(id=3178, text=' source', logprob=-0.28100586, special=False), generated_text=None, details=None)
+        TextGenerationStreamOutput(token=TokenElement(id=273, text=' and', logprob=-0.5961914, special=False), generated_text=None, details=None)
+        TextGenerationStreamOutput(token=TokenElement(id=3426, text=' built', logprob=-1.9423828, special=False), generated_text=None, details=None)
+        TextGenerationStreamOutput(token=TokenElement(id=271, text=' to', logprob=-1.4121094, special=False), generated_text=None, details=None)
+        TextGenerationStreamOutput(token=TokenElement(id=314, text=' be', logprob=-1.5224609, special=False), generated_text=None, details=None)
+        TextGenerationStreamOutput(token=TokenElement(id=1833, text=' easy', logprob=-2.1132812, special=False), generated_text=None, details=None)
+        TextGenerationStreamOutput(token=TokenElement(id=271, text=' to', logprob=-0.08520508, special=False), generated_text=None, details=None)
+        TextGenerationStreamOutput(token=TokenElement(id=745, text=' use', logprob=-0.39453125, special=False), generated_text=None, details=None)
+        TextGenerationStreamOutput(token=TokenElement(
             id=25,
             text='.',
             logprob=-0.5703125,
             special=False),
             generated_text='100% open source and built to be easy to use.',
-            details=StreamDetails(finish_reason=<FinishReason.Length: 'length'>, generated_tokens=12, seed=None)
+            details=TextGenerationStreamDetails(finish_reason='length', generated_tokens=12, seed=None)
         )
         ```
         """
-        # NOTE: Text-generation integration is taken from the text-generation-inference project. It has more features
-        # like input/output validation (if Pydantic is installed). See `_text_generation.py` header for more details.
-
         if decoder_input_details and not details:
             warnings.warn(
                 "`decoder_input_details=True` has been passed to the server but `details=False` is set meaning that"
@@ -1463,34 +1759,38 @@ class InferenceClient:
             )
             decoder_input_details = False
 
-        # Validate parameters
-        parameters = TextGenerationParameters(
-            best_of=best_of,
-            details=details,
-            do_sample=do_sample,
-            max_new_tokens=max_new_tokens,
-            repetition_penalty=repetition_penalty,
-            return_full_text=return_full_text,
-            seed=seed,
-            stop=stop_sequences if stop_sequences is not None else [],
-            temperature=temperature,
-            top_k=top_k,
-            top_p=top_p,
-            truncate=truncate,
-            typical_p=typical_p,
-            watermark=watermark,
-            decoder_input_details=decoder_input_details,
-        )
-        request = TextGenerationRequest(inputs=prompt, stream=stream, parameters=parameters)
-        payload = asdict(request)
+        # Build payload
+        payload = {
+            "inputs": prompt,
+            "parameters": {
+                "best_of": best_of,
+                "decoder_input_details": decoder_input_details,
+                "details": details,
+                "do_sample": do_sample,
+                "max_new_tokens": max_new_tokens,
+                "repetition_penalty": repetition_penalty,
+                "return_full_text": return_full_text,
+                "seed": seed,
+                "stop": stop_sequences if stop_sequences is not None else [],
+                "temperature": temperature,
+                "top_k": top_k,
+                "top_p": top_p,
+                "truncate": truncate,
+                "typical_p": typical_p,
+                "watermark": watermark,
+            },
+            "stream": stream,
+        }
 
         # Remove some parameters if not a TGI server
         if not _is_tgi_server(model):
+            parameters: Dict = payload["parameters"]  # type: ignore [assignment]
+
             ignored_parameters = []
-            for key in "watermark", "stop", "details", "decoder_input_details", "best_of":
-                if payload["parameters"][key] is not None:
+            for key in "watermark", "details", "decoder_input_details", "best_of", "stop":
+                if parameters[key] is not None:
                     ignored_parameters.append(key)
-                del payload["parameters"][key]
+                del parameters[key]
             if len(ignored_parameters) > 0:
                 warnings.warn(
                     "API endpoint/model for text-generation is not served via TGI. Ignoring parameters"
@@ -1542,8 +1842,8 @@ class InferenceClient:
         if stream:
             return _stream_text_generation_response(bytes_output, details)  # type: ignore
 
-        data = _bytes_to_dict(bytes_output)[0]
-        return TextGenerationResponse(**data) if details else data["generated_text"]
+        data = _bytes_to_dict(bytes_output)[0]  # type: ignore[arg-type]
+        return TextGenerationOutput.parse_obj_as_instance(data) if details else data["generated_text"]
 
     def text_to_image(
         self,
diff --git a/src/huggingface_hub/inference/_common.py b/src/huggingface_hub/inference/_common.py
index c4367e75..01ea3257 100644
--- a/src/huggingface_hub/inference/_common.py
+++ b/src/huggingface_hub/inference/_common.py
@@ -18,6 +18,7 @@ import base64
 import io
 import json
 import logging
+import time
 from contextlib import contextmanager
 from dataclasses import dataclass
 from pathlib import Path
@@ -32,12 +33,24 @@ from typing import (
     Iterable,
     List,
     Literal,
+    NoReturn,
     Optional,
     Set,
     Union,
     overload,
 )
 
+from requests import HTTPError
+
+from huggingface_hub.errors import (
+    GenerationError,
+    IncompleteGenerationError,
+    OverloadedError,
+    TextGenerationError,
+    UnknownError,
+    ValidationError,
+)
+
 from ..constants import ENDPOINT
 from ..utils import (
     build_hf_headers,
@@ -47,7 +60,12 @@ from ..utils import (
     is_numpy_available,
     is_pillow_available,
 )
-from ._text_generation import TextGenerationStreamResponse, _parse_text_generation_error
+from ._generated.types import (
+    ChatCompletionStreamOutput,
+    ChatCompletionStreamOutputChoice,
+    ChatCompletionStreamOutputDelta,
+    TextGenerationStreamOutput,
+)
 
 
 if TYPE_CHECKING:
@@ -250,48 +268,125 @@ def _bytes_to_image(content: bytes) -> "Image":
 
 def _stream_text_generation_response(
     bytes_output_as_lines: Iterable[bytes], details: bool
-) -> Union[Iterable[str], Iterable[TextGenerationStreamResponse]]:
+) -> Union[Iterable[str], Iterable[TextGenerationStreamOutput]]:
+    """Used in `InferenceClient.text_generation`."""
     # Parse ServerSentEvents
     for byte_payload in bytes_output_as_lines:
-        # Skip line
-        if byte_payload == b"\n":
-            continue
-
-        payload = byte_payload.decode("utf-8")
-
-        # Event data
-        if payload.startswith("data:"):
-            # Decode payload
-            json_payload = json.loads(payload.lstrip("data:").rstrip("/n"))
-            # Either an error as being returned
-            if json_payload.get("error") is not None:
-                raise _parse_text_generation_error(json_payload["error"], json_payload.get("error_type"))
-            # Or parse token payload
-            output = TextGenerationStreamResponse(**json_payload)
-            yield output.token.text if not details else output
+        output = _format_text_generation_stream_output(byte_payload, details)
+        if output is not None:
+            yield output
 
 
 async def _async_stream_text_generation_response(
     bytes_output_as_lines: AsyncIterable[bytes], details: bool
-) -> Union[AsyncIterable[str], AsyncIterable[TextGenerationStreamResponse]]:
+) -> Union[AsyncIterable[str], AsyncIterable[TextGenerationStreamOutput]]:
+    """Used in `AsyncInferenceClient.text_generation`."""
     # Parse ServerSentEvents
     async for byte_payload in bytes_output_as_lines:
-        # Skip line
-        if byte_payload == b"\n":
-            continue
+        output = _format_text_generation_stream_output(byte_payload, details)
+        if output is not None:
+            yield output
+
+
+def _format_text_generation_stream_output(
+    byte_payload: bytes, details: bool
+) -> Optional[Union[str, TextGenerationStreamOutput]]:
+    if not byte_payload.startswith(b"data:"):
+        return None  # empty line
+
+    # Decode payload
+    payload = byte_payload.decode("utf-8")
+    json_payload = json.loads(payload.lstrip("data:").rstrip("/n"))
+
+    # Either an error as being returned
+    if json_payload.get("error") is not None:
+        raise _parse_text_generation_error(json_payload["error"], json_payload.get("error_type"))
+
+    # Or parse token payload
+    output = TextGenerationStreamOutput.parse_obj_as_instance(json_payload)
+    return output.token.text if not details else output
+
+
+def _stream_chat_completion_response_from_text_generation(
+    text_generation_output: Iterable[TextGenerationStreamOutput],
+) -> Iterable[ChatCompletionStreamOutput]:
+    """Used in `InferenceClient.chat_completion`."""
+    created = int(time.time())
+    for item in text_generation_output:
+        yield _format_chat_completion_stream_output_from_text_generation(item, created)
+
+
+async def _async_stream_chat_completion_response_from_text_generation(
+    text_generation_output: AsyncIterable[TextGenerationStreamOutput],
+) -> AsyncIterable[ChatCompletionStreamOutput]:
+    """Used in `AsyncInferenceClient.chat_completion`."""
+    created = int(time.time())
+    async for item in text_generation_output:
+        yield _format_chat_completion_stream_output_from_text_generation(item, created)
+
+
+def _format_chat_completion_stream_output_from_text_generation(
+    item: TextGenerationStreamOutput, created: int
+) -> ChatCompletionStreamOutput:
+    if item.details is None:
+        # new token generated => return delta
+        return ChatCompletionStreamOutput(
+            choices=[
+                ChatCompletionStreamOutputChoice(
+                    delta=ChatCompletionStreamOutputDelta(
+                        role="assistant",
+                        content=item.token.text,
+                    ),
+                    finish_reason=None,
+                    index=0,
+                )
+            ],
+            created=created,
+        )
+    else:
+        # generation is completed => return finish reason
+        return ChatCompletionStreamOutput(
+            choices=[
+                ChatCompletionStreamOutputChoice(
+                    delta=ChatCompletionStreamOutputDelta(),
+                    finish_reason=item.details.finish_reason,
+                    index=0,
+                )
+            ],
+            created=created,
+        )
+
+
+def _stream_chat_completion_response_from_bytes(
+    bytes_lines: Iterable[bytes],
+) -> Iterable[ChatCompletionStreamOutput]:
+    """Used in `InferenceClient.chat_completion` if model is served with TGI."""
+    for item in bytes_lines:
+        output = _format_chat_completion_stream_output_from_text_generation_from_bytes(item)
+        if output is not None:
+            yield output
+
 
-        payload = byte_payload.decode("utf-8")
+async def _async_stream_chat_completion_response_from_bytes(
+    bytes_lines: AsyncIterable[bytes],
+) -> AsyncIterable[ChatCompletionStreamOutput]:
+    """Used in `AsyncInferenceClient.chat_completion`."""
+    async for item in bytes_lines:
+        output = _format_chat_completion_stream_output_from_text_generation_from_bytes(item)
+        if output is not None:
+            yield output
 
-        # Event data
-        if payload.startswith("data:"):
-            # Decode payload
-            json_payload = json.loads(payload.lstrip("data:").rstrip("/n"))
-            # Either an error as being returned
-            if json_payload.get("error") is not None:
-                raise _parse_text_generation_error(json_payload["error"], json_payload.get("error_type"))
-            # Or parse token payload
-            output = TextGenerationStreamResponse(**json_payload)
-            yield output.token.text if not details else output
+
+def _format_chat_completion_stream_output_from_text_generation_from_bytes(
+    byte_payload: bytes,
+) -> Optional[ChatCompletionStreamOutput]:
+    if not byte_payload.startswith(b"data:"):
+        return None  # empty line
+
+    # Decode payload
+    payload = byte_payload.decode("utf-8")
+    json_payload = json.loads(payload.lstrip("data:").rstrip("/n"))
+    return ChatCompletionStreamOutput.parse_obj_as_instance(json_payload)
 
 
 async def _async_yield_from(client: "ClientSession", response: "ClientResponse") -> AsyncIterable[bytes]:
@@ -311,6 +406,10 @@ async def _async_yield_from(client: "ClientSession", response: "ClientResponse")
 # default API with a warning message. We remember for each model if it's a TGI server
 # or not using `_NON_TGI_SERVERS` global variable.
 #
+# In addition, TGI servers have a built-in API route for chat-completion, which is not
+# available on the default API. We use this route to provide a more consistent behavior
+# when available.
+#
 # For more details, see https://github.com/huggingface/text-generation-inference and
 # https://huggingface.co/docs/api-inference/detailed_parameters#text-generation-task.
 
@@ -323,3 +422,61 @@ def _set_as_non_tgi(model: Optional[str]) -> None:
 
 def _is_tgi_server(model: Optional[str]) -> bool:
     return model not in _NON_TGI_SERVERS
+
+
+_NON_CHAT_COMPLETION_SERVER: Set[str] = set()
+
+
+def _set_as_non_chat_completion_server(model: str) -> None:
+    print("Set as non chat completion", model)
+    _NON_CHAT_COMPLETION_SERVER.add(model)
+
+
+def _is_chat_completion_server(model: str) -> bool:
+    return model not in _NON_CHAT_COMPLETION_SERVER
+
+
+# TEXT GENERATION ERRORS
+# ----------------------
+# Text-generation errors are parsed separately to handle as much as possible the errors returned by the text generation
+# inference project (https://github.com/huggingface/text-generation-inference).
+# ----------------------
+
+
+def raise_text_generation_error(http_error: HTTPError) -> NoReturn:
+    """
+    Try to parse text-generation-inference error message and raise HTTPError in any case.
+
+    Args:
+        error (`HTTPError`):
+            The HTTPError that have been raised.
+    """
+    # Try to parse a Text Generation Inference error
+
+    try:
+        # Hacky way to retrieve payload in case of aiohttp error
+        payload = getattr(http_error, "response_error_payload", None) or http_error.response.json()
+        error = payload.get("error")
+        error_type = payload.get("error_type")
+    except Exception:  # no payload
+        raise http_error
+
+    # If error_type => more information than `hf_raise_for_status`
+    if error_type is not None:
+        exception = _parse_text_generation_error(error, error_type)
+        raise exception from http_error
+
+    # Otherwise, fallback to default error
+    raise http_error
+
+
+def _parse_text_generation_error(error: Optional[str], error_type: Optional[str]) -> TextGenerationError:
+    if error_type == "generation":
+        return GenerationError(error)  # type: ignore
+    if error_type == "incomplete_generation":
+        return IncompleteGenerationError(error)  # type: ignore
+    if error_type == "overloaded":
+        return OverloadedError(error)  # type: ignore
+    if error_type == "validation":
+        return ValidationError(error)  # type: ignore
+    return UnknownError(error)  # type: ignore
diff --git a/src/huggingface_hub/inference/_generated/_async_client.py b/src/huggingface_hub/inference/_generated/_async_client.py
index 12d268bd..1b695b71 100644
--- a/src/huggingface_hub/inference/_generated/_async_client.py
+++ b/src/huggingface_hub/inference/_generated/_async_client.py
@@ -23,7 +23,6 @@ import base64
 import logging
 import time
 import warnings
-from dataclasses import asdict
 from typing import (
     TYPE_CHECKING,
     Any,
@@ -44,6 +43,8 @@ from huggingface_hub.inference._common import (
     TASKS_EXPECTING_IMAGES,
     ContentT,
     ModelStatus,
+    _async_stream_chat_completion_response_from_bytes,
+    _async_stream_chat_completion_response_from_text_generation,
     _async_stream_text_generation_response,
     _b64_encode,
     _b64_to_image,
@@ -52,14 +53,21 @@ from huggingface_hub.inference._common import (
     _bytes_to_list,
     _fetch_recommended_models,
     _import_numpy,
+    _is_chat_completion_server,
     _is_tgi_server,
     _open_as_binary,
+    _set_as_non_chat_completion_server,
     _set_as_non_tgi,
+    raise_text_generation_error,
 )
 from huggingface_hub.inference._generated.types import (
     AudioClassificationOutputElement,
     AudioToAudioOutputElement,
     AutomaticSpeechRecognitionOutput,
+    ChatCompletionOutput,
+    ChatCompletionOutputChoice,
+    ChatCompletionOutputChoiceMessage,
+    ChatCompletionStreamOutput,
     DocumentQuestionAnsweringOutputElement,
     FillMaskOutputElement,
     ImageClassificationOutputElement,
@@ -70,19 +78,15 @@ from huggingface_hub.inference._generated.types import (
     SummarizationOutput,
     TableQuestionAnsweringOutputElement,
     TextClassificationOutputElement,
+    TextGenerationOutput,
+    TextGenerationStreamOutput,
     TokenClassificationOutputElement,
     TranslationOutput,
     VisualQuestionAnsweringOutputElement,
     ZeroShotClassificationOutputElement,
     ZeroShotImageClassificationOutputElement,
 )
-from huggingface_hub.inference._text_generation import (
-    TextGenerationParameters,
-    TextGenerationRequest,
-    TextGenerationResponse,
-    TextGenerationStreamResponse,
-    raise_text_generation_error,
-)
+from huggingface_hub.inference._templating import render_chat_prompt
 from huggingface_hub.inference._types import (
     ConversationalOutput,  # soon to be removed
 )
@@ -112,9 +116,9 @@ class AsyncInferenceClient:
             The model to run inference with. Can be a model id hosted on the Hugging Face Hub, e.g. `bigcode/starcoder`
             or a URL to a deployed Inference Endpoint. Defaults to None, in which case a recommended model is
             automatically selected for the task.
-        token (`str`, *optional*):
-            Hugging Face token. Will default to the locally saved token. Pass `token=False` if you don't want to send
-            your token to the server.
+        token (`str` or `bool`, *optional*):
+            Hugging Face token. Will default to the locally saved token if not provided.
+            Pass `token=False` if you don't want to send your token to the server.
         timeout (`float`, `optional`):
             The maximum number of seconds to wait for a response from the server. Loading a new model in Inference
             API can take up to several minutes. Defaults to None, meaning it will loop until the server is available.
@@ -134,6 +138,7 @@ class AsyncInferenceClient:
         cookies: Optional[Dict[str, str]] = None,
     ) -> None:
         self.model: Optional[str] = model
+        self.token: Union[str, bool, None] = token
         self.headers = CaseInsensitiveDict(build_hf_headers(token=token))  # contains 'authorization' + 'user-agent'
         if headers is not None:
             self.headers.update(headers)
@@ -152,11 +157,10 @@ class AsyncInferenceClient:
         model: Optional[str] = None,
         task: Optional[str] = None,
         stream: Literal[False] = ...,
-    ) -> bytes:
-        pass
+    ) -> bytes: ...
 
     @overload
-    async def post(
+    async def post(  # type: ignore[misc]
         self,
         *,
         json: Optional[Union[str, Dict, List]] = None,
@@ -164,8 +168,18 @@ class AsyncInferenceClient:
         model: Optional[str] = None,
         task: Optional[str] = None,
         stream: Literal[True] = ...,
-    ) -> AsyncIterable[bytes]:
-        pass
+    ) -> AsyncIterable[bytes]: ...
+
+    @overload
+    async def post(
+        self,
+        *,
+        json: Optional[Union[str, Dict, List]] = None,
+        data: Optional[ContentT] = None,
+        model: Optional[str] = None,
+        task: Optional[str] = None,
+        stream: bool = False,
+    ) -> Union[bytes, AsyncIterable[bytes]]: ...
 
     async def post(
         self,
@@ -394,6 +408,260 @@ class AsyncInferenceClient:
         response = await self.post(data=audio, model=model, task="automatic-speech-recognition")
         return AutomaticSpeechRecognitionOutput.parse_obj_as_instance(response)
 
+    @overload
+    async def chat_completion(  # type: ignore
+        self,
+        messages: List[Dict[str, str]],
+        *,
+        model: Optional[str] = None,
+        stream: Literal[False] = False,
+        max_tokens: int = 20,
+        seed: Optional[int] = None,
+        stop: Optional[Union[List[str], str]] = None,
+        temperature: float = 1.0,
+        top_p: Optional[float] = None,
+    ) -> ChatCompletionOutput: ...
+
+    @overload
+    async def chat_completion(  # type: ignore
+        self,
+        messages: List[Dict[str, str]],
+        *,
+        model: Optional[str] = None,
+        stream: Literal[True] = True,
+        max_tokens: int = 20,
+        seed: Optional[int] = None,
+        stop: Optional[Union[List[str], str]] = None,
+        temperature: float = 1.0,
+        top_p: Optional[float] = None,
+    ) -> AsyncIterable[ChatCompletionStreamOutput]: ...
+
+    @overload
+    async def chat_completion(
+        self,
+        messages: List[Dict[str, str]],
+        *,
+        model: Optional[str] = None,
+        stream: bool = False,
+        max_tokens: int = 20,
+        seed: Optional[int] = None,
+        stop: Optional[Union[List[str], str]] = None,
+        temperature: float = 1.0,
+        top_p: Optional[float] = None,
+    ) -> Union[ChatCompletionOutput, AsyncIterable[ChatCompletionStreamOutput]]: ...
+
+    async def chat_completion(
+        self,
+        messages: List[Dict[str, str]],
+        *,
+        model: Optional[str] = None,
+        stream: bool = False,
+        max_tokens: int = 20,
+        seed: Optional[int] = None,
+        stop: Optional[Union[List[str], str]] = None,
+        temperature: float = 1.0,
+        top_p: Optional[float] = None,
+    ) -> Union[ChatCompletionOutput, AsyncIterable[ChatCompletionStreamOutput]]:
+        """
+        A method for completing conversations using a specified language model.
+
+        <Tip>
+
+        If the model is served by a server supporting chat-completion, the method will directly call the server's
+        `/v1/chat/completions` endpoint. If the server does not support chat-completion, the method will render the
+        chat template client-side based on the information fetched from the Hub API. In this case, you will need to
+        have `minijinja` template engine installed. Run `pip install "huggingface_hub[inference]"` or `pip install minijinja`
+        to install it.
+
+        </Tip>
+
+        Args:
+            messages (List[Union[`SystemMessage`, `UserMessage`, `AssistantMessage`]]):
+                Conversation history consisting of roles and content pairs.
+            model (`str`, *optional*):
+                The model to use for chat-completion. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed
+                Inference Endpoint. If not provided, the default recommended model for chat-based text-generation will be used.
+                See https://huggingface.co/tasks/text-generation for more details.
+            frequency_penalty (`float`, optional):
+                Penalizes new tokens based on their existing frequency
+                in the text so far. Range: [-2.0, 2.0]. Defaults to 0.0.
+            max_tokens (`int`, optional):
+                Maximum number of tokens allowed in the response. Defaults to 20.
+            seed (Optional[`int`], optional):
+                Seed for reproducible control flow. Defaults to None.
+            stop (Optional[`str`], optional):
+                Up to four strings which trigger the end of the response.
+                Defaults to None.
+            stream (`bool`, optional):
+                Enable realtime streaming of responses. Defaults to False.
+            temperature (`float`, optional):
+                Controls randomness of the generations. Lower values ensure
+                less random completions. Range: [0, 2]. Defaults to 1.0.
+            top_p (`float`, optional):
+                Fraction of the most likely next words to sample from.
+                Must be between 0 and 1. Defaults to 1.0.
+
+        Returns:
+            `Union[ChatCompletionOutput, Iterable[ChatCompletionStreamOutput]]`:
+            Generated text returned from the server:
+            - if `stream=False`, the generated text is returned as a [`ChatCompletionOutput`] (default).
+            - if `stream=True`, the generated text is returned token by token as a sequence of [`ChatCompletionStreamOutput`].
+
+        Raises:
+            [`InferenceTimeoutError`]:
+                If the model is unavailable or the request times out.
+            `aiohttp.ClientResponseError`:
+                If the request fails with an HTTP error status code other than HTTP 503.
+
+        Example:
+        ```py
+        # Must be run in an async context
+        >>> from huggingface_hub import AsyncInferenceClient
+        >>> messages = [{"role": "user", "content": "What is the capital of France?"}]
+        >>> client = AsyncInferenceClient("HuggingFaceH4/zephyr-7b-beta")
+        >>> await client.chat_completion(messages, max_tokens=100)
+        ChatCompletionOutput(
+            choices=[
+                ChatCompletionOutputChoice(
+                    finish_reason='eos_token',
+                    index=0,
+                    message=ChatCompletionOutputChoiceMessage(
+                        content='The capital of France is Paris. The official name of the city is "Ville de Paris" (City of Paris) and the name of the country\'s governing body, which is located in Paris, is "La République française" (The French Republic). \nI hope that helps! Let me know if you need any further information.'
+                    )
+                )
+            ],
+            created=1710498360
+        )
+
+        >>> async for token in await client.chat_completion(messages, max_tokens=10, stream=True):
+        ...     print(token)
+        ChatCompletionStreamOutput(choices=[ChatCompletionStreamOutputChoice(delta=ChatCompletionStreamOutputDelta(content='The', role='assistant'), index=0, finish_reason=None)], created=1710498504)
+        ChatCompletionStreamOutput(choices=[ChatCompletionStreamOutputChoice(delta=ChatCompletionStreamOutputDelta(content=' capital', role='assistant'), index=0, finish_reason=None)], created=1710498504)
+        (...)
+        ChatCompletionStreamOutput(choices=[ChatCompletionStreamOutputChoice(delta=ChatCompletionStreamOutputDelta(content=' may', role='assistant'), index=0, finish_reason=None)], created=1710498504)
+        ChatCompletionStreamOutput(choices=[ChatCompletionStreamOutputChoice(delta=ChatCompletionStreamOutputDelta(content=None, role=None), index=0, finish_reason='length')], created=1710498504)
+        ```
+        """
+        # determine model
+        model = model or self.model or self.get_recommended_model("text-generation")
+
+        if _is_chat_completion_server(model):
+            # First, let's consider the server has a `/v1/chat/completions` endpoint.
+            # If that's the case, we don't have to render the chat template client-side.
+            model_url = self._resolve_url(model) + "/v1/chat/completions"
+
+            try:
+                data = await self.post(
+                    model=model_url,
+                    json=dict(
+                        model="tgi",  # random string
+                        messages=messages,
+                        max_tokens=max_tokens,
+                        seed=seed,
+                        stop=stop,
+                        temperature=temperature,
+                        top_p=top_p,
+                        stream=stream,
+                    ),
+                    stream=stream,
+                )
+            except _import_aiohttp().ClientResponseError:
+                # Let's consider the server is not a chat completion server.
+                # Then we call again `chat_completion` which will render the chat template client side.
+                # (can be HTTP 500, HTTP 400, HTTP 404 depending on the server)
+                _set_as_non_chat_completion_server(model)
+                return await self.chat_completion(
+                    messages=messages,
+                    model=model,
+                    stream=stream,
+                    max_tokens=max_tokens,
+                    seed=seed,
+                    stop=stop,
+                    temperature=temperature,
+                    top_p=top_p,
+                )
+
+            if stream:
+                return _async_stream_chat_completion_response_from_bytes(data)  # type: ignore[arg-type]
+
+            return ChatCompletionOutput.parse_obj_as_instance(data)  # type: ignore[arg-type]
+
+        # At this point, we know the server is not a chat completion server.
+        # We need to render the chat template client side based on the information we can fetch from
+        # the Hub API.
+
+        model_id = None
+        if model.startswith(("http://", "https://")):
+            # If URL, we need to know which model is served. This is not always possible.
+            # A workaround is to list the user Inference Endpoints and check if one of them correspond to the model URL.
+            # If not, we raise an error.
+            # TODO: fix when we have a proper API for this (at least for Inference Endpoints)
+            # TODO: what if Sagemaker URL?
+            # TODO: what if Azure URL?
+            from ..hf_api import HfApi
+
+            for endpoint in HfApi(token=self.token).list_inference_endpoints():
+                if endpoint.url == model:
+                    model_id = endpoint.repository
+                    break
+        else:
+            model_id = model
+
+        if model_id is None:
+            # If we don't have the model ID, we can't fetch the chat template.
+            # We raise an error.
+            raise ValueError(
+                "Request can't be processed as the model ID can't be inferred from model URL. "
+                "This is needed to fetch the chat template from the Hub since the model is not "
+                "served with a Chat-completion API."
+            )
+
+        # fetch chat template + tokens
+        prompt = render_chat_prompt(model_id=model_id, token=self.token, messages=messages)
+
+        # generate response
+        stop_sequences = [stop] if isinstance(stop, str) else stop
+        text_generation_output = await self.text_generation(
+            prompt=prompt,
+            details=True,
+            stream=stream,
+            model=model,
+            max_new_tokens=max_tokens,
+            seed=seed,
+            stop_sequences=stop_sequences,
+            temperature=temperature,
+            top_p=top_p,
+        )
+
+        created = int(time.time())
+
+        if stream:
+            return _async_stream_chat_completion_response_from_text_generation(text_generation_output)  # type: ignore [arg-type]
+
+        if isinstance(text_generation_output, TextGenerationOutput):
+            # General use case => format ChatCompletionOutput from text generation details
+            content: str = text_generation_output.generated_text
+            finish_reason: str = text_generation_output.details.finish_reason  # type: ignore[union-attr]
+        else:
+            # Corner case: if server doesn't support details (e.g. if not a TGI server), we only receive an output string.
+            # In such a case, `finish_reason` is set to `"unk"`.
+            content = text_generation_output  # type: ignore[assignment]
+            finish_reason = "unk"
+
+        return ChatCompletionOutput(
+            created=created,
+            choices=[
+                ChatCompletionOutputChoice(
+                    finish_reason=finish_reason,  # type: ignore
+                    index=0,
+                    message=ChatCompletionOutputChoiceMessage(
+                        content=content,
+                        role="assistant",
+                    ),
+                )
+            ],
+        )
+
     async def conversational(
         self,
         text: str,
@@ -406,6 +674,13 @@ class AsyncInferenceClient:
         """
         Generate conversational responses based on the given input text (i.e. chat with the API).
 
+        <Tip warning={true}>
+
+        [`InferenceClient.conversational`] API is deprecated and will be removed in a future release. Please use
+        [`InferenceClient.chat_completion`] instead.
+
+        </Tip>
+
         Args:
             text (`str`):
                 The last input from the user in the conversation.
@@ -446,6 +721,11 @@ class AsyncInferenceClient:
         ... )
         ```
         """
+        warnings.warn(
+            "'InferenceClient.conversational' is deprecated and will be removed starting from huggingface_hub>=0.25. "
+            "Please use the more appropriate 'InferenceClient.chat_completion' API instead.",
+            FutureWarning,
+        )
         payload: Dict[str, Any] = {"inputs": {"text": text}}
         if generated_responses is not None:
             payload["inputs"]["generated_responses"] = generated_responses
@@ -1253,7 +1533,7 @@ class AsyncInferenceClient:
         truncate: Optional[int] = None,
         typical_p: Optional[float] = None,
         watermark: bool = False,
-    ) -> TextGenerationResponse: ...
+    ) -> TextGenerationOutput: ...
 
     @overload
     async def text_generation(  # type: ignore
@@ -1279,7 +1559,7 @@ class AsyncInferenceClient:
     ) -> AsyncIterable[str]: ...
 
     @overload
-    async def text_generation(
+    async def text_generation(  # type: ignore
         self,
         prompt: str,
         *,
@@ -1299,7 +1579,30 @@ class AsyncInferenceClient:
         truncate: Optional[int] = None,
         typical_p: Optional[float] = None,
         watermark: bool = False,
-    ) -> AsyncIterable[TextGenerationStreamResponse]: ...
+    ) -> AsyncIterable[TextGenerationStreamOutput]: ...
+
+    @overload
+    async def text_generation(
+        self,
+        prompt: str,
+        *,
+        details: Literal[True] = ...,
+        stream: bool = ...,
+        model: Optional[str] = None,
+        do_sample: bool = False,
+        max_new_tokens: int = 20,
+        best_of: Optional[int] = None,
+        repetition_penalty: Optional[float] = None,
+        return_full_text: bool = False,
+        seed: Optional[int] = None,
+        stop_sequences: Optional[List[str]] = None,
+        temperature: Optional[float] = None,
+        top_k: Optional[int] = None,
+        top_p: Optional[float] = None,
+        truncate: Optional[int] = None,
+        typical_p: Optional[float] = None,
+        watermark: bool = False,
+    ) -> Union[TextGenerationOutput, AsyncIterable[TextGenerationStreamOutput]]: ...
 
     async def text_generation(
         self,
@@ -1322,13 +1625,10 @@ class AsyncInferenceClient:
         typical_p: Optional[float] = None,
         watermark: bool = False,
         decoder_input_details: bool = False,
-    ) -> Union[str, TextGenerationResponse, AsyncIterable[str], AsyncIterable[TextGenerationStreamResponse]]:
+    ) -> Union[str, TextGenerationOutput, AsyncIterable[str], AsyncIterable[TextGenerationStreamOutput]]:
         """
         Given a prompt, generate the following text.
 
-        It is recommended to have Pydantic installed in order to get inputs validated. This is preferable as it allow
-        early failures.
-
         API endpoint is supposed to run with the `text-generation-inference` backend (TGI). This backend is the
         go-to solution to run large language models at scale. However, for some smaller models (e.g. "gpt2") the
         default `transformers` + `api-inference` solution is still in use. Both approaches have very similar APIs, but
@@ -1386,12 +1686,12 @@ class AsyncInferenceClient:
                 into account. Defaults to `False`.
 
         Returns:
-            `Union[str, TextGenerationResponse, Iterable[str], Iterable[TextGenerationStreamResponse]]`:
+            `Union[str, TextGenerationOutput, Iterable[str], Iterable[TextGenerationStreamOutput]]`:
             Generated text returned from the server:
             - if `stream=False` and `details=False`, the generated text is returned as a `str` (default)
             - if `stream=True` and `details=False`, the generated text is returned token by token as a `Iterable[str]`
-            - if `stream=False` and `details=True`, the generated text is returned with more details as a [`~huggingface_hub.inference._text_generation.TextGenerationResponse`]
-            - if `details=True` and `stream=True`, the generated text is returned token by token as a iterable of [`~huggingface_hub.inference._text_generation.TextGenerationStreamResponse`]
+            - if `stream=False` and `details=True`, the generated text is returned with more details as a [`~huggingface_hub.TextGenerationOutput`]
+            - if `details=True` and `stream=True`, the generated text is returned token by token as a iterable of [`~huggingface_hub.TextGenerationStreamOutput`]
 
         Raises:
             `ValidationError`:
@@ -1429,23 +1729,23 @@ class AsyncInferenceClient:
 
         # Case 3: get more details about the generation process.
         >>> await client.text_generation("The huggingface_hub library is ", max_new_tokens=12, details=True)
-        TextGenerationResponse(
+        TextGenerationOutput(
             generated_text='100% open source and built to be easy to use.',
-            details=Details(
-                finish_reason=<FinishReason.Length: 'length'>,
+            details=TextGenerationDetails(
+                finish_reason='length',
                 generated_tokens=12,
                 seed=None,
                 prefill=[
-                    InputToken(id=487, text='The', logprob=None),
-                    InputToken(id=53789, text=' hugging', logprob=-13.171875),
+                    TextGenerationPrefillToken(id=487, text='The', logprob=None),
+                    TextGenerationPrefillToken(id=53789, text=' hugging', logprob=-13.171875),
                     (...)
-                    InputToken(id=204, text=' ', logprob=-7.0390625)
+                    TextGenerationPrefillToken(id=204, text=' ', logprob=-7.0390625)
                 ],
                 tokens=[
-                    Token(id=1425, text='100', logprob=-1.0175781, special=False),
-                    Token(id=16, text='%', logprob=-0.0463562, special=False),
+                    TokenElement(id=1425, text='100', logprob=-1.0175781, special=False),
+                    TokenElement(id=16, text='%', logprob=-0.0463562, special=False),
                     (...)
-                    Token(id=25, text='.', logprob=-0.5703125, special=False)
+                    TokenElement(id=25, text='.', logprob=-0.5703125, special=False)
                 ],
                 best_of_sequences=None
             )
@@ -1456,30 +1756,27 @@ class AsyncInferenceClient:
         >>> async for details in await client.text_generation("The huggingface_hub library is ", max_new_tokens=12, details=True, stream=True):
         ...     print(details)
         ...
-        TextGenerationStreamResponse(token=Token(id=1425, text='100', logprob=-1.0175781, special=False), generated_text=None, details=None)
-        TextGenerationStreamResponse(token=Token(id=16, text='%', logprob=-0.0463562, special=False), generated_text=None, details=None)
-        TextGenerationStreamResponse(token=Token(id=1314, text=' open', logprob=-1.3359375, special=False), generated_text=None, details=None)
-        TextGenerationStreamResponse(token=Token(id=3178, text=' source', logprob=-0.28100586, special=False), generated_text=None, details=None)
-        TextGenerationStreamResponse(token=Token(id=273, text=' and', logprob=-0.5961914, special=False), generated_text=None, details=None)
-        TextGenerationStreamResponse(token=Token(id=3426, text=' built', logprob=-1.9423828, special=False), generated_text=None, details=None)
-        TextGenerationStreamResponse(token=Token(id=271, text=' to', logprob=-1.4121094, special=False), generated_text=None, details=None)
-        TextGenerationStreamResponse(token=Token(id=314, text=' be', logprob=-1.5224609, special=False), generated_text=None, details=None)
-        TextGenerationStreamResponse(token=Token(id=1833, text=' easy', logprob=-2.1132812, special=False), generated_text=None, details=None)
-        TextGenerationStreamResponse(token=Token(id=271, text=' to', logprob=-0.08520508, special=False), generated_text=None, details=None)
-        TextGenerationStreamResponse(token=Token(id=745, text=' use', logprob=-0.39453125, special=False), generated_text=None, details=None)
-        TextGenerationStreamResponse(token=Token(
+        TextGenerationStreamOutput(token=TokenElement(id=1425, text='100', logprob=-1.0175781, special=False), generated_text=None, details=None)
+        TextGenerationStreamOutput(token=TokenElement(id=16, text='%', logprob=-0.0463562, special=False), generated_text=None, details=None)
+        TextGenerationStreamOutput(token=TokenElement(id=1314, text=' open', logprob=-1.3359375, special=False), generated_text=None, details=None)
+        TextGenerationStreamOutput(token=TokenElement(id=3178, text=' source', logprob=-0.28100586, special=False), generated_text=None, details=None)
+        TextGenerationStreamOutput(token=TokenElement(id=273, text=' and', logprob=-0.5961914, special=False), generated_text=None, details=None)
+        TextGenerationStreamOutput(token=TokenElement(id=3426, text=' built', logprob=-1.9423828, special=False), generated_text=None, details=None)
+        TextGenerationStreamOutput(token=TokenElement(id=271, text=' to', logprob=-1.4121094, special=False), generated_text=None, details=None)
+        TextGenerationStreamOutput(token=TokenElement(id=314, text=' be', logprob=-1.5224609, special=False), generated_text=None, details=None)
+        TextGenerationStreamOutput(token=TokenElement(id=1833, text=' easy', logprob=-2.1132812, special=False), generated_text=None, details=None)
+        TextGenerationStreamOutput(token=TokenElement(id=271, text=' to', logprob=-0.08520508, special=False), generated_text=None, details=None)
+        TextGenerationStreamOutput(token=TokenElement(id=745, text=' use', logprob=-0.39453125, special=False), generated_text=None, details=None)
+        TextGenerationStreamOutput(token=TokenElement(
             id=25,
             text='.',
             logprob=-0.5703125,
             special=False),
             generated_text='100% open source and built to be easy to use.',
-            details=StreamDetails(finish_reason=<FinishReason.Length: 'length'>, generated_tokens=12, seed=None)
+            details=TextGenerationStreamDetails(finish_reason='length', generated_tokens=12, seed=None)
         )
         ```
         """
-        # NOTE: Text-generation integration is taken from the text-generation-inference project. It has more features
-        # like input/output validation (if Pydantic is installed). See `_text_generation.py` header for more details.
-
         if decoder_input_details and not details:
             warnings.warn(
                 "`decoder_input_details=True` has been passed to the server but `details=False` is set meaning that"
@@ -1487,34 +1784,38 @@ class AsyncInferenceClient:
             )
             decoder_input_details = False
 
-        # Validate parameters
-        parameters = TextGenerationParameters(
-            best_of=best_of,
-            details=details,
-            do_sample=do_sample,
-            max_new_tokens=max_new_tokens,
-            repetition_penalty=repetition_penalty,
-            return_full_text=return_full_text,
-            seed=seed,
-            stop=stop_sequences if stop_sequences is not None else [],
-            temperature=temperature,
-            top_k=top_k,
-            top_p=top_p,
-            truncate=truncate,
-            typical_p=typical_p,
-            watermark=watermark,
-            decoder_input_details=decoder_input_details,
-        )
-        request = TextGenerationRequest(inputs=prompt, stream=stream, parameters=parameters)
-        payload = asdict(request)
+        # Build payload
+        payload = {
+            "inputs": prompt,
+            "parameters": {
+                "best_of": best_of,
+                "decoder_input_details": decoder_input_details,
+                "details": details,
+                "do_sample": do_sample,
+                "max_new_tokens": max_new_tokens,
+                "repetition_penalty": repetition_penalty,
+                "return_full_text": return_full_text,
+                "seed": seed,
+                "stop": stop_sequences if stop_sequences is not None else [],
+                "temperature": temperature,
+                "top_k": top_k,
+                "top_p": top_p,
+                "truncate": truncate,
+                "typical_p": typical_p,
+                "watermark": watermark,
+            },
+            "stream": stream,
+        }
 
         # Remove some parameters if not a TGI server
         if not _is_tgi_server(model):
+            parameters: Dict = payload["parameters"]  # type: ignore [assignment]
+
             ignored_parameters = []
-            for key in "watermark", "stop", "details", "decoder_input_details", "best_of":
-                if payload["parameters"][key] is not None:
+            for key in "watermark", "details", "decoder_input_details", "best_of", "stop":
+                if parameters[key] is not None:
                     ignored_parameters.append(key)
-                del payload["parameters"][key]
+                del parameters[key]
             if len(ignored_parameters) > 0:
                 warnings.warn(
                     "API endpoint/model for text-generation is not served via TGI. Ignoring parameters"
@@ -1567,8 +1868,8 @@ class AsyncInferenceClient:
         if stream:
             return _async_stream_text_generation_response(bytes_output, details)  # type: ignore
 
-        data = _bytes_to_dict(bytes_output)[0]
-        return TextGenerationResponse(**data) if details else data["generated_text"]
+        data = _bytes_to_dict(bytes_output)[0]  # type: ignore[arg-type]
+        return TextGenerationOutput.parse_obj_as_instance(data) if details else data["generated_text"]
 
     async def text_to_image(
         self,
diff --git a/src/huggingface_hub/inference/_generated/types/__init__.py b/src/huggingface_hub/inference/_generated/types/__init__.py
index 3ca3d111..ece102c4 100644
--- a/src/huggingface_hub/inference/_generated/types/__init__.py
+++ b/src/huggingface_hub/inference/_generated/types/__init__.py
@@ -17,6 +17,16 @@ from .automatic_speech_recognition import (
     AutomaticSpeechRecognitionParameters,
 )
 from .base import BaseInferenceType
+from .chat_completion import (
+    ChatCompletionInput,
+    ChatCompletionInputMessage,
+    ChatCompletionOutput,
+    ChatCompletionOutputChoice,
+    ChatCompletionOutputChoiceMessage,
+    ChatCompletionStreamOutput,
+    ChatCompletionStreamOutputChoice,
+    ChatCompletionStreamOutputDelta,
+)
 from .depth_estimation import DepthEstimationInput, DepthEstimationOutput
 from .document_question_answering import (
     DocumentQuestionAnsweringInput,
@@ -56,13 +66,15 @@ from .table_question_answering import (
 from .text2text_generation import Text2TextGenerationInput, Text2TextGenerationOutput, Text2TextGenerationParameters
 from .text_classification import TextClassificationInput, TextClassificationOutputElement, TextClassificationParameters
 from .text_generation import (
-    PrefillToken,
     TextGenerationInput,
     TextGenerationOutput,
     TextGenerationOutputDetails,
+    TextGenerationOutputSequenceDetails,
+    TextGenerationOutputToken,
     TextGenerationParameters,
-    TextGenerationSequenceDetails,
-    Token,
+    TextGenerationPrefillToken,
+    TextGenerationStreamDetails,
+    TextGenerationStreamOutput,
 )
 from .text_to_audio import TextToAudioGenerationParameters, TextToAudioInput, TextToAudioOutput, TextToAudioParameters
 from .text_to_image import TextToImageInput, TextToImageOutput, TextToImageParameters, TextToImageTargetSize
diff --git a/src/huggingface_hub/inference/_generated/types/chat_completion.py b/src/huggingface_hub/inference/_generated/types/chat_completion.py
new file mode 100644
index 00000000..43e24f81
--- /dev/null
+++ b/src/huggingface_hub/inference/_generated/types/chat_completion.py
@@ -0,0 +1,106 @@
+# Inference code generated from the JSON schema spec in @huggingface/tasks.
+#
+# See:
+#   - script: https://github.com/huggingface/huggingface.js/blob/main/packages/tasks/scripts/inference-codegen.ts
+#   - specs:  https://github.com/huggingface/huggingface.js/tree/main/packages/tasks/src/tasks.
+from dataclasses import dataclass
+from typing import List, Literal, Optional, Union
+
+from .base import BaseInferenceType
+
+
+ChatCompletionMessageRole = Literal["assistant", "system", "user"]
+
+
+@dataclass
+class ChatCompletionInputMessage(BaseInferenceType):
+    content: str
+    """The content of the message."""
+    role: "ChatCompletionMessageRole"
+
+
+@dataclass
+class ChatCompletionInput(BaseInferenceType):
+    """Inputs for ChatCompletion inference"""
+
+    messages: List[ChatCompletionInputMessage]
+    frequency_penalty: Optional[float] = None
+    """Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing
+    frequency in the text so far, decreasing the model's likelihood to repeat the same line
+    verbatim.
+    """
+    max_tokens: Optional[int] = None
+    """The maximum number of tokens that can be generated in the chat completion."""
+    seed: Optional[int] = None
+    """The random sampling seed."""
+    stop: Optional[Union[List[str], str]] = None
+    """Stop generating tokens if a stop token is generated."""
+    stream: Optional[bool] = None
+    """If set, partial message deltas will be sent."""
+    temperature: Optional[float] = None
+    """The value used to modulate the logits distribution."""
+    top_p: Optional[float] = None
+    """If set to < 1, only the smallest set of most probable tokens with probabilities that add
+    up to `top_p` or higher are kept for generation.
+    """
+
+
+ChatCompletionFinishReason = Literal["length", "eos_token", "stop_sequence"]
+
+
+@dataclass
+class ChatCompletionOutputChoiceMessage(BaseInferenceType):
+    content: str
+    """The content of the chat completion message."""
+    role: "ChatCompletionMessageRole"
+
+
+@dataclass
+class ChatCompletionOutputChoice(BaseInferenceType):
+    finish_reason: "ChatCompletionFinishReason"
+    """The reason why the generation was stopped."""
+    index: int
+    """The index of the choice in the list of choices."""
+    message: ChatCompletionOutputChoiceMessage
+
+
+@dataclass
+class ChatCompletionOutput(BaseInferenceType):
+    """Outputs for Chat Completion inference"""
+
+    choices: List[ChatCompletionOutputChoice]
+    """A list of chat completion choices."""
+    created: int
+    """The Unix timestamp (in seconds) of when the chat completion was created."""
+
+
+@dataclass
+class ChatCompletionStreamOutputDelta(BaseInferenceType):
+    """A chat completion delta generated by streamed model responses."""
+
+    content: Optional[str] = None
+    """The contents of the chunk message."""
+    role: Optional[str] = None
+    """The role of the author of this message."""
+
+
+@dataclass
+class ChatCompletionStreamOutputChoice(BaseInferenceType):
+    delta: ChatCompletionStreamOutputDelta
+    """A chat completion delta generated by streamed model responses."""
+    index: int
+    """The index of the choice in the list of choices."""
+    finish_reason: Optional["ChatCompletionFinishReason"] = None
+    """The reason why the generation was stopped."""
+
+
+@dataclass
+class ChatCompletionStreamOutput(BaseInferenceType):
+    """Chat Completion Stream Output"""
+
+    choices: List[ChatCompletionStreamOutputChoice]
+    """A list of chat completion choices."""
+    created: int
+    """The Unix timestamp (in seconds) of when the chat completion was created. Each chunk has
+    the same timestamp.
+    """
diff --git a/src/huggingface_hub/inference/_generated/types/text_generation.py b/src/huggingface_hub/inference/_generated/types/text_generation.py
index dca7c6e9..28669850 100644
--- a/src/huggingface_hub/inference/_generated/types/text_generation.py
+++ b/src/huggingface_hub/inference/_generated/types/text_generation.py
@@ -63,13 +63,15 @@ class TextGenerationInput(BaseInferenceType):
     """The text to initialize generation with"""
     parameters: Optional[TextGenerationParameters] = None
     """Additional inference parameters"""
+    stream: Optional[bool] = None
+    """Whether to stream output tokens"""
 
 
-FinishReason = Literal["length", "eos_token", "stop_sequence"]
+TextGenerationFinishReason = Literal["length", "eos_token", "stop_sequence"]
 
 
 @dataclass
-class PrefillToken(BaseInferenceType):
+class TextGenerationPrefillToken(BaseInferenceType):
     id: int
     logprob: float
     text: str
@@ -77,45 +79,50 @@ class PrefillToken(BaseInferenceType):
 
 
 @dataclass
-class Token(BaseInferenceType):
+class TextGenerationOutputToken(BaseInferenceType):
+    """Generated token."""
+
     id: int
-    logprob: float
     special: bool
     """Whether or not that token is a special one"""
     text: str
     """The text associated with that token"""
+    logprob: Optional[float] = None
 
 
 @dataclass
-class TextGenerationSequenceDetails(BaseInferenceType):
-    finish_reason: "FinishReason"
-    """The reason why the generation was stopped."""
-    generated_text: int
+class TextGenerationOutputSequenceDetails(BaseInferenceType):
+    finish_reason: "TextGenerationFinishReason"
+    generated_text: str
     """The generated text"""
     generated_tokens: int
     """The number of generated tokens"""
-    prefill: List[PrefillToken]
-    tokens: List[Token]
+    prefill: List[TextGenerationPrefillToken]
+    tokens: List[TextGenerationOutputToken]
     """The generated tokens and associated details"""
     seed: Optional[int] = None
     """The random seed used for generation"""
+    top_tokens: Optional[List[List[TextGenerationOutputToken]]] = None
+    """Most likely tokens"""
 
 
 @dataclass
 class TextGenerationOutputDetails(BaseInferenceType):
     """When enabled, details about the generation"""
 
-    finish_reason: "FinishReason"
+    finish_reason: "TextGenerationFinishReason"
     """The reason why the generation was stopped."""
     generated_tokens: int
     """The number of generated tokens"""
-    prefill: List[PrefillToken]
-    tokens: List[Token]
+    prefill: List[TextGenerationPrefillToken]
+    tokens: List[TextGenerationOutputToken]
     """The generated tokens and associated details"""
-    best_of_sequences: Optional[List[TextGenerationSequenceDetails]] = None
+    best_of_sequences: Optional[List[TextGenerationOutputSequenceDetails]] = None
     """Details about additional sequences when best_of is provided"""
     seed: Optional[int] = None
     """The random seed used for generation"""
+    top_tokens: Optional[List[List[TextGenerationOutputToken]]] = None
+    """Most likely tokens"""
 
 
 @dataclass
@@ -126,3 +133,29 @@ class TextGenerationOutput(BaseInferenceType):
     """The generated text"""
     details: Optional[TextGenerationOutputDetails] = None
     """When enabled, details about the generation"""
+
+
+@dataclass
+class TextGenerationStreamDetails(BaseInferenceType):
+    """Generation details. Only available when the generation is finished."""
+
+    finish_reason: "TextGenerationFinishReason"
+    """The reason why the generation was stopped."""
+    generated_tokens: int
+    """The number of generated tokens"""
+    seed: int
+    """The random seed used for generation"""
+
+
+@dataclass
+class TextGenerationStreamOutput(BaseInferenceType):
+    """Text Generation Stream Output"""
+
+    token: TextGenerationOutputToken
+    """Generated token."""
+    details: Optional[TextGenerationStreamDetails] = None
+    """Generation details. Only available when the generation is finished."""
+    generated_text: Optional[str] = None
+    """The complete generated text. Only available when the generation is finished."""
+    index: Optional[int] = None
+    """The token index within the stream. Optional to support older clients that omit it."""
diff --git a/src/huggingface_hub/inference/_templating.py b/src/huggingface_hub/inference/_templating.py
new file mode 100644
index 00000000..10c349c2
--- /dev/null
+++ b/src/huggingface_hub/inference/_templating.py
@@ -0,0 +1,105 @@
+from functools import lru_cache
+from typing import Callable, Dict, List, Optional, Union
+
+from ..utils import HfHubHTTPError, RepositoryNotFoundError, is_minijinja_available
+
+
+class TemplateError(Exception):
+    """Any error raised while trying to fetch or render a chat template."""
+
+
+def _import_minijinja():
+    if not is_minijinja_available():
+        raise ImportError("Cannot render template. Please install minijinja using `pip install minijinja`.")
+    import minijinja  # noqa: F401
+
+    return minijinja
+
+
+def render_chat_prompt(
+    *,
+    model_id: str,
+    messages: List[Dict[str, str]],
+    token: Union[str, bool, None] = None,
+    add_generation_prompt: bool = True,
+    **kwargs,
+) -> str:
+    """Render a chat prompt using a model's chat template.
+
+    Args:
+        model_id (`str`):
+            The model id.
+        messages (`List[Dict[str, str]]`):
+            The list of messages to render.
+        token (`str` or `bool`, *optional*):
+            Hugging Face token. Will default to the locally saved token if not provided.
+
+    Returns:
+        `str`: The rendered chat prompt.
+
+    Raises:
+        `TemplateError`: If there's any issue while fetching, compiling or rendering the chat template.
+    """
+    minijinja = _import_minijinja()
+    template = _fetch_and_compile_template(model_id=model_id, token=token)
+
+    try:
+        return template(messages=messages, add_generation_prompt=add_generation_prompt, **kwargs)
+    except minijinja.TemplateError as e:
+        raise TemplateError(f"Error while trying to render chat prompt for model '{model_id}': {e}") from e
+
+
+@lru_cache  # TODO: lru_cache for raised exceptions
+def _fetch_and_compile_template(*, model_id: str, token: Union[str, None]) -> Callable:
+    """Fetch and compile a model's chat template.
+
+    Method is cached to avoid fetching the same model's config multiple times.
+
+    Args:
+        model_id (`str`):
+            The model id.
+        token (`str` or `bool`, *optional*):
+            Hugging Face token. Will default to the locally saved token if not provided.
+
+    Returns:
+        `Callable`: A callable that takes a list of messages and returns the rendered chat prompt.
+    """
+    from huggingface_hub.hf_api import HfApi
+
+    minijinja = _import_minijinja()
+
+    # 1. fetch config from API
+    try:
+        config = HfApi(token=token).model_info(model_id).config
+    except RepositoryNotFoundError as e:
+        raise TemplateError(f"Cannot render chat template: model '{model_id}' not found.") from e
+    except HfHubHTTPError as e:
+        raise TemplateError(f"Error while trying to fetch chat template for model '{model_id}': {e}") from e
+
+    # 2. check config validity
+    if config is None:
+        raise TemplateError(f"Config not found for model '{model_id}'.")
+    tokenizer_config = config.get("tokenizer_config")
+    if tokenizer_config is None:
+        raise TemplateError(f"Tokenizer config not found for model '{model_id}'.")
+    if tokenizer_config.get("chat_template") is None:
+        raise TemplateError(f"Chat template not found in tokenizer_config for model '{model_id}'.")
+    chat_template = tokenizer_config["chat_template"]
+    if not isinstance(chat_template, str):
+        raise TemplateError(f"Chat template must be a string, not '{type(chat_template)}' (model: {model_id}).")
+
+    special_tokens: Dict[str, Optional[str]] = {}
+    for key, value in tokenizer_config.items():
+        if "token" in key:
+            if isinstance(value, str):
+                special_tokens[key] = value
+            elif isinstance(value, dict) and value.get("__type") == "AddedToken":
+                special_tokens[key] = value.get("content")
+
+    # 3. compile template and return
+    env = minijinja.Environment()
+    try:
+        env.add_template("chat_template", chat_template)
+    except minijinja.TemplateError as e:
+        raise TemplateError(f"Error while trying to compile chat template for model '{model_id}': {e}") from e
+    return lambda **kwargs: env.render_template("chat_template", **kwargs, **special_tokens)
diff --git a/src/huggingface_hub/inference/_text_generation.py b/src/huggingface_hub/inference/_text_generation.py
deleted file mode 100644
index 570d54c4..00000000
--- a/src/huggingface_hub/inference/_text_generation.py
+++ /dev/null
@@ -1,535 +0,0 @@
-# coding=utf-8
-# Copyright 2023-present, the HuggingFace Inc. team.
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#
-# Original implementation taken from the `text-generation` Python client (see https://pypi.org/project/text-generation/
-# and https://github.com/huggingface/text-generation-inference/tree/main/clients/python)
-#
-# Changes compared to original implementation:
-# - use pydantic.dataclasses instead of BaseModel
-# - default to Python's dataclasses if Pydantic is not installed (same implementation but no validation)
-# - added default values for all parameters (not needed in BaseModel but dataclasses yes)
-# - integrated in `huggingface_hub.InferenceClient``
-# - added `stream: bool` and `details: bool` in the `text_generation` method instead of having different methods for each use case
-import warnings
-from dataclasses import field
-from enum import Enum
-from typing import List, NoReturn, Optional
-
-from requests import HTTPError
-
-from huggingface_hub.errors import (
-    GenerationError,
-    IncompleteGenerationError,
-    OverloadedError,
-    TextGenerationError,
-    UnknownError,
-    ValidationError,
-)
-
-from ..utils import is_pydantic_available
-
-
-if is_pydantic_available():
-    from pydantic import validator as pydantic_validator
-    from pydantic.dataclasses import dataclass
-
-    def validator(*args, **kwargs):
-        # Pydantic v1's `@validator` is deprecated in favor of `@field_validator`. In order to support both pydantic v1
-        # and v2 without changing the logic, we catch the warning message in pydantic v2 and ignore it. If we want to
-        # support pydantic v3 in the future, we will drop support for pydantic v1 and use `pydantic.field_validator`
-        # correctly.
-        #
-        # Related:
-        # - https://docs.pydantic.dev/latest/migration/#changes-to-validators
-        # - https://github.com/huggingface/huggingface_hub/pull/1837
-        with warnings.catch_warnings():
-            warnings.filterwarnings("ignore", message="Pydantic V1 style `@validator` validators are deprecated.")
-            return pydantic_validator(*args, **kwargs)
-else:
-    # No validation if Pydantic is not installed
-    from dataclasses import dataclass  # type: ignore
-
-    def validator(x):  # type: ignore
-        return lambda y: y
-
-
-@dataclass
-class TextGenerationParameters:
-    """
-    Parameters for text generation.
-
-    Args:
-        do_sample (`bool`, *optional*):
-            Activate logits sampling. Defaults to False.
-        max_new_tokens (`int`, *optional*):
-            Maximum number of generated tokens. Defaults to 20.
-        repetition_penalty (`Optional[float]`, *optional*):
-            The parameter for repetition penalty. A value of 1.0 means no penalty. See [this paper](https://arxiv.org/pdf/1909.05858.pdf)
-            for more details. Defaults to None.
-        return_full_text (`bool`, *optional*):
-            Whether to prepend the prompt to the generated text. Defaults to False.
-        stop (`List[str]`, *optional*):
-            Stop generating tokens if a member of `stop_sequences` is generated. Defaults to an empty list.
-        seed (`Optional[int]`, *optional*):
-            Random sampling seed. Defaults to None.
-        temperature (`Optional[float]`, *optional*):
-            The value used to modulate the logits distribution. Defaults to None.
-        top_k (`Optional[int]`, *optional*):
-            The number of highest probability vocabulary tokens to keep for top-k-filtering. Defaults to None.
-        top_p (`Optional[float]`, *optional*):
-            If set to a value less than 1, only the smallest set of most probable tokens with probabilities that add up
-            to `top_p` or higher are kept for generation. Defaults to None.
-        truncate (`Optional[int]`, *optional*):
-            Truncate input tokens to the given size. Defaults to None.
-        typical_p (`Optional[float]`, *optional*):
-            Typical Decoding mass. See [Typical Decoding for Natural Language Generation](https://arxiv.org/abs/2202.00666)
-            for more information. Defaults to None.
-        best_of (`Optional[int]`, *optional*):
-            Generate `best_of` sequences and return the one with the highest token logprobs. Defaults to None.
-        watermark (`bool`, *optional*):
-            Watermarking with [A Watermark for Large Language Models](https://arxiv.org/abs/2301.10226). Defaults to False.
-        details (`bool`, *optional*):
-            Get generation details. Defaults to False.
-        decoder_input_details (`bool`, *optional*):
-            Get decoder input token logprobs and ids. Defaults to False.
-    """
-
-    # Activate logits sampling
-    do_sample: bool = False
-    # Maximum number of generated tokens
-    max_new_tokens: int = 20
-    # The parameter for repetition penalty. 1.0 means no penalty.
-    # See [this paper](https://arxiv.org/pdf/1909.05858.pdf) for more details.
-    repetition_penalty: Optional[float] = None
-    # Whether to prepend the prompt to the generated text
-    return_full_text: bool = False
-    # Stop generating tokens if a member of `stop_sequences` is generated
-    stop: List[str] = field(default_factory=lambda: [])
-    # Random sampling seed
-    seed: Optional[int] = None
-    # The value used to module the logits distribution.
-    temperature: Optional[float] = None
-    # The number of highest probability vocabulary tokens to keep for top-k-filtering.
-    top_k: Optional[int] = None
-    # If set to < 1, only the smallest set of most probable tokens with probabilities that add up to `top_p` or
-    # higher are kept for generation.
-    top_p: Optional[float] = None
-    # truncate inputs tokens to the given size
-    truncate: Optional[int] = None
-    # Typical Decoding mass
-    # See [Typical Decoding for Natural Language Generation](https://arxiv.org/abs/2202.00666) for more information
-    typical_p: Optional[float] = None
-    # Generate best_of sequences and return the one if the highest token logprobs
-    best_of: Optional[int] = None
-    # Watermarking with [A Watermark for Large Language Models](https://arxiv.org/abs/2301.10226)
-    watermark: bool = False
-    # Get generation details
-    details: bool = False
-    # Get decoder input token logprobs and ids
-    decoder_input_details: bool = False
-
-    @validator("best_of")
-    def valid_best_of(cls, field_value, values):
-        if field_value is not None:
-            if field_value <= 0:
-                raise ValueError("`best_of` must be strictly positive")
-            if field_value > 1 and values["seed"] is not None:
-                raise ValueError("`seed` must not be set when `best_of` is > 1")
-            sampling = (
-                values["do_sample"]
-                | (values["temperature"] is not None)
-                | (values["top_k"] is not None)
-                | (values["top_p"] is not None)
-                | (values["typical_p"] is not None)
-            )
-            if field_value > 1 and not sampling:
-                raise ValueError("you must use sampling when `best_of` is > 1")
-
-        return field_value
-
-    @validator("repetition_penalty")
-    def valid_repetition_penalty(cls, v):
-        if v is not None and v <= 0:
-            raise ValueError("`repetition_penalty` must be strictly positive")
-        return v
-
-    @validator("seed")
-    def valid_seed(cls, v):
-        if v is not None and v < 0:
-            raise ValueError("`seed` must be positive")
-        return v
-
-    @validator("temperature")
-    def valid_temp(cls, v):
-        if v is not None and v <= 0:
-            raise ValueError("`temperature` must be strictly positive")
-        return v
-
-    @validator("top_k")
-    def valid_top_k(cls, v):
-        if v is not None and v <= 0:
-            raise ValueError("`top_k` must be strictly positive")
-        return v
-
-    @validator("top_p")
-    def valid_top_p(cls, v):
-        if v is not None and (v <= 0 or v >= 1.0):
-            raise ValueError("`top_p` must be > 0.0 and < 1.0")
-        return v
-
-    @validator("truncate")
-    def valid_truncate(cls, v):
-        if v is not None and v <= 0:
-            raise ValueError("`truncate` must be strictly positive")
-        return v
-
-    @validator("typical_p")
-    def valid_typical_p(cls, v):
-        if v is not None and (v <= 0 or v >= 1.0):
-            raise ValueError("`typical_p` must be > 0.0 and < 1.0")
-        return v
-
-
-@dataclass
-class TextGenerationRequest:
-    """
-    Request object for text generation (only for internal use).
-
-    Args:
-        inputs (`str`):
-            The prompt for text generation.
-        parameters (`Optional[TextGenerationParameters]`, *optional*):
-            Generation parameters.
-        stream (`bool`, *optional*):
-            Whether to stream output tokens. Defaults to False.
-    """
-
-    # Prompt
-    inputs: str
-    # Generation parameters
-    parameters: Optional[TextGenerationParameters] = None
-    # Whether to stream output tokens
-    stream: bool = False
-
-    @validator("inputs")
-    def valid_input(cls, v):
-        if not v:
-            raise ValueError("`inputs` cannot be empty")
-        return v
-
-    @validator("stream")
-    def valid_best_of_stream(cls, field_value, values):
-        parameters = values["parameters"]
-        if parameters is not None and parameters.best_of is not None and parameters.best_of > 1 and field_value:
-            raise ValueError("`best_of` != 1 is not supported when `stream` == True")
-        return field_value
-
-    def __post_init__(self):
-        if not is_pydantic_available():
-            # If pydantic is not installed, we need to instantiate the nested dataclasses manually
-            if self.parameters is not None and isinstance(self.parameters, dict):
-                self.parameters = TextGenerationParameters(**self.parameters)
-
-
-# Decoder input tokens
-@dataclass
-class InputToken:
-    """
-    Represents an input token.
-
-    Args:
-        id (`int`):
-            Token ID from the model tokenizer.
-        text (`str`):
-            Token text.
-        logprob (`float` or `None`):
-            Log probability of the token. Optional since the logprob of the first token cannot be computed.
-    """
-
-    # Token ID from the model tokenizer
-    id: int
-    # Token text
-    text: str
-    # Logprob
-    # Optional since the logprob of the first token cannot be computed
-    logprob: Optional[float] = None
-
-
-# Generated tokens
-@dataclass
-class Token:
-    """
-    Represents a token.
-
-    Args:
-        id (`int`):
-            Token ID from the model tokenizer.
-        text (`str`):
-            Token text.
-        logprob (`float`):
-            Log probability of the token.
-        special (`bool`):
-            Indicates whether the token is a special token. It can be used to ignore
-            tokens when concatenating.
-    """
-
-    # Token ID from the model tokenizer
-    id: int
-    # Token text
-    text: str
-    # Logprob
-    logprob: float
-    # Is the token a special token
-    # Can be used to ignore tokens when concatenating
-    special: bool
-
-
-# Generation finish reason
-class FinishReason(str, Enum):
-    # number of generated tokens == `max_new_tokens`
-    Length = "length"
-    # the model generated its end of sequence token
-    EndOfSequenceToken = "eos_token"
-    # the model generated a text included in `stop_sequences`
-    StopSequence = "stop_sequence"
-
-
-# Additional sequences when using the `best_of` parameter
-@dataclass
-class BestOfSequence:
-    """
-    Represents a best-of sequence generated during text generation.
-
-    Args:
-        generated_text (`str`):
-            The generated text.
-        finish_reason (`FinishReason`):
-            The reason for the generation to finish, represented by a `FinishReason` value.
-        generated_tokens (`int`):
-            The number of generated tokens in the sequence.
-        seed (`Optional[int]`):
-            The sampling seed if sampling was activated.
-        prefill (`List[InputToken]`):
-            The decoder input tokens. Empty if `decoder_input_details` is False. Defaults to an empty list.
-        tokens (`List[Token]`):
-            The generated tokens. Defaults to an empty list.
-    """
-
-    # Generated text
-    generated_text: str
-    # Generation finish reason
-    finish_reason: FinishReason
-    # Number of generated tokens
-    generated_tokens: int
-    # Sampling seed if sampling was activated
-    seed: Optional[int] = None
-    # Decoder input tokens, empty if decoder_input_details is False
-    prefill: List[InputToken] = field(default_factory=lambda: [])
-    # Generated tokens
-    tokens: List[Token] = field(default_factory=lambda: [])
-
-    def __post_init__(self):
-        if not is_pydantic_available():
-            # If pydantic is not installed, we need to instantiate the nested dataclasses manually
-            self.prefill = [
-                InputToken(**input_token) if isinstance(input_token, dict) else input_token
-                for input_token in self.prefill
-            ]
-            self.tokens = [Token(**token) if isinstance(token, dict) else token for token in self.tokens]
-
-
-# `generate` details
-@dataclass
-class Details:
-    """
-    Represents details of a text generation.
-
-    Args:
-        finish_reason (`FinishReason`):
-            The reason for the generation to finish, represented by a `FinishReason` value.
-        generated_tokens (`int`):
-            The number of generated tokens.
-        seed (`Optional[int]`):
-            The sampling seed if sampling was activated.
-        prefill (`List[InputToken]`, *optional*):
-            The decoder input tokens. Empty if `decoder_input_details` is False. Defaults to an empty list.
-        tokens (`List[Token]`):
-            The generated tokens. Defaults to an empty list.
-        best_of_sequences (`Optional[List[BestOfSequence]]`):
-            Additional sequences when using the `best_of` parameter.
-    """
-
-    # Generation finish reason
-    finish_reason: FinishReason
-    # Number of generated tokens
-    generated_tokens: int
-    # Sampling seed if sampling was activated
-    seed: Optional[int] = None
-    # Decoder input tokens, empty if decoder_input_details is False
-    prefill: List[InputToken] = field(default_factory=lambda: [])
-    # Generated tokens
-    tokens: List[Token] = field(default_factory=lambda: [])
-    # Additional sequences when using the `best_of` parameter
-    best_of_sequences: Optional[List[BestOfSequence]] = None
-
-    def __post_init__(self):
-        if not is_pydantic_available():
-            # If pydantic is not installed, we need to instantiate the nested dataclasses manually
-            self.prefill = [
-                InputToken(**input_token) if isinstance(input_token, dict) else input_token
-                for input_token in self.prefill
-            ]
-            self.tokens = [Token(**token) if isinstance(token, dict) else token for token in self.tokens]
-            if self.best_of_sequences is not None:
-                self.best_of_sequences = [
-                    BestOfSequence(**best_of_sequence) if isinstance(best_of_sequence, dict) else best_of_sequence
-                    for best_of_sequence in self.best_of_sequences
-                ]
-
-
-# `generate` return value
-@dataclass
-class TextGenerationResponse:
-    """
-    Represents a response for text generation.
-
-    Only returned when `details=True`, otherwise a string is returned.
-
-    Args:
-        generated_text (`str`):
-            The generated text.
-        details (`Optional[Details]`):
-            Generation details. Returned only if `details=True` is sent to the server.
-    """
-
-    # Generated text
-    generated_text: str
-    # Generation details
-    details: Optional[Details] = None
-
-    def __post_init__(self):
-        if not is_pydantic_available():
-            # If pydantic is not installed, we need to instantiate the nested dataclasses manually
-            if self.details is not None and isinstance(self.details, dict):
-                self.details = Details(**self.details)
-
-
-# `generate_stream` details
-@dataclass
-class StreamDetails:
-    """
-    Represents details of a text generation stream.
-
-    Args:
-        finish_reason (`FinishReason`):
-            The reason for the generation to finish, represented by a `FinishReason` value.
-        generated_tokens (`int`):
-            The number of generated tokens.
-        seed (`Optional[int]`):
-            The sampling seed if sampling was activated.
-    """
-
-    # Generation finish reason
-    finish_reason: FinishReason
-    # Number of generated tokens
-    generated_tokens: int
-    # Sampling seed if sampling was activated
-    seed: Optional[int] = None
-
-
-# `generate_stream` return value
-@dataclass
-class TextGenerationStreamResponse:
-    """
-    Represents a response for streaming text generation.
-
-    Only returned when `details=True` and `stream=True`.
-
-    Args:
-        token (`Token`):
-            The generated token.
-        index (`Optional[int]`, *optional*):
-            The token index within the stream. Optional to support older clients that omit it.
-        generated_text (`Optional[str]`, *optional*):
-            The complete generated text. Only available when the generation is finished.
-        details (`Optional[StreamDetails]`, *optional*):
-            Generation details. Only available when the generation is finished.
-    """
-
-    # Generated token
-    token: Token
-    # The token index within the stream
-    # Optional to support  older clients that omit it.
-    index: Optional[int] = None
-    # Complete generated text
-    # Only available when the generation is finished
-    generated_text: Optional[str] = None
-    # Generation details
-    # Only available when the generation is finished
-    details: Optional[StreamDetails] = None
-
-    def __post_init__(self):
-        if not is_pydantic_available():
-            # If pydantic is not installed, we need to instantiate the nested dataclasses manually
-            if isinstance(self.token, dict):
-                self.token = Token(**self.token)
-            if self.details is not None and isinstance(self.details, dict):
-                self.details = StreamDetails(**self.details)
-
-
-# TEXT GENERATION ERRORS
-# ----------------------
-# Text-generation errors are parsed separately to handle as much as possible the errors returned by the text generation
-# inference project (https://github.com/huggingface/text-generation-inference).
-# ----------------------
-
-
-def raise_text_generation_error(http_error: HTTPError) -> NoReturn:
-    """
-    Try to parse text-generation-inference error message and raise HTTPError in any case.
-
-    Args:
-        error (`HTTPError`):
-            The HTTPError that have been raised.
-    """
-    # Try to parse a Text Generation Inference error
-
-    try:
-        # Hacky way to retrieve payload in case of aiohttp error
-        payload = getattr(http_error, "response_error_payload", None) or http_error.response.json()
-        error = payload.get("error")
-        error_type = payload.get("error_type")
-    except Exception:  # no payload
-        raise http_error
-
-    # If error_type => more information than `hf_raise_for_status`
-    if error_type is not None:
-        exception = _parse_text_generation_error(error, error_type)
-        raise exception from http_error
-
-    # Otherwise, fallback to default error
-    raise http_error
-
-
-def _parse_text_generation_error(error: Optional[str], error_type: Optional[str]) -> TextGenerationError:
-    if error_type == "generation":
-        return GenerationError(error)  # type: ignore
-    if error_type == "incomplete_generation":
-        return IncompleteGenerationError(error)  # type: ignore
-    if error_type == "overloaded":
-        return OverloadedError(error)  # type: ignore
-    if error_type == "validation":
-        return ValidationError(error)  # type: ignore
-    return UnknownError(error)  # type: ignore
diff --git a/src/huggingface_hub/utils/__init__.py b/src/huggingface_hub/utils/__init__.py
index cf52fd6f..0a3a0a32 100644
--- a/src/huggingface_hub/utils/__init__.py
+++ b/src/huggingface_hub/utils/__init__.py
@@ -60,6 +60,7 @@ from ._runtime import (
     get_hf_hub_version,
     get_hf_transfer_version,
     get_jinja_version,
+    get_minijinja_version,
     get_numpy_version,
     get_pillow_version,
     get_pydantic_version,
@@ -76,6 +77,7 @@ from ._runtime import (
     is_graphviz_available,
     is_hf_transfer_available,
     is_jinja_available,
+    is_minijinja_available,
     is_notebook,
     is_numpy_available,
     is_package_available,
diff --git a/src/huggingface_hub/utils/_runtime.py b/src/huggingface_hub/utils/_runtime.py
index 5a11aedc..a57926f0 100644
--- a/src/huggingface_hub/utils/_runtime.py
+++ b/src/huggingface_hub/utils/_runtime.py
@@ -35,6 +35,7 @@ _CANDIDATES = {
     "graphviz": {"graphviz"},
     "hf_transfer": {"hf_transfer"},
     "jinja": {"Jinja2"},
+    "minijinja": {"minijinja"},
     "numpy": {"numpy"},
     "pillow": {"Pillow"},
     "pydantic": {"pydantic"},
@@ -139,6 +140,15 @@ def get_hf_transfer_version() -> str:
     return _get_version("hf_transfer")
 
 
+# Minijinja
+def is_minijinja_available() -> bool:
+    return is_package_available("minijinja")
+
+
+def get_minijinja_version() -> str:
+    return _get_version("minijinja")
+
+
 # Numpy
 def is_numpy_available() -> bool:
     return is_package_available("numpy")
diff --git a/utils/generate_async_inference_client.py b/utils/generate_async_inference_client.py
index c5fe7584..e787ddad 100644
--- a/utils/generate_async_inference_client.py
+++ b/utils/generate_async_inference_client.py
@@ -49,9 +49,12 @@ def generate_async_client_code(code: str) -> str:
     # Make all tasks-method async
     code = _make_tasks_methods_async(code)
 
-    # Adapt text_generation error catching
+    # Adapt text_generation to async
     code = _adapt_text_generation_to_async(code)
 
+    # Adapt chat_completion to async
+    code = _adapt_chat_completion_to_async(code)
+
     # Update some docstrings
     code = _rename_HTTPError_to_ClientResponseError_in_docstring(code)
     code = _update_examples_in_public_methods(code)
@@ -294,16 +297,61 @@ def _adapt_text_generation_to_async(code: str) -> str:
     )
 
     # Await recursive call
-    code = code.replace("return self.text_generation", "return await self.text_generation")
+    code = code.replace(
+        "return self.text_generation",
+        "return await self.text_generation",
+    )
+    code = code.replace(
+        "return self.chat_completion",
+        "return await self.chat_completion",
+    )
 
     # Update return types: Iterable -> AsyncIterable
-    code = code.replace(") -> Iterable[str]:", ") -> AsyncIterable[str]:")
     code = code.replace(
-        ") -> Iterable[TextGenerationStreamResponse]:", ") -> AsyncIterable[TextGenerationStreamResponse]:"
+        ") -> Iterable[str]:",
+        ") -> AsyncIterable[str]:",
+    )
+    code = code.replace(
+        ") -> Union[bytes, Iterable[bytes]]:",
+        ") -> Union[bytes, AsyncIterable[bytes]]:",
+    )
+    code = code.replace(
+        ") -> Iterable[TextGenerationStreamOutput]:",
+        ") -> AsyncIterable[TextGenerationStreamOutput]:",
     )
     code = code.replace(
-        ") -> Union[str, TextGenerationResponse, Iterable[str], Iterable[TextGenerationStreamResponse]]:",
-        ") -> Union[str, TextGenerationResponse, AsyncIterable[str], AsyncIterable[TextGenerationStreamResponse]]:",
+        ") -> Union[TextGenerationOutput, Iterable[TextGenerationStreamOutput]]:",
+        ") -> Union[TextGenerationOutput, AsyncIterable[TextGenerationStreamOutput]]:",
+    )
+    code = code.replace(
+        ") -> Union[str, TextGenerationOutput, Iterable[str], Iterable[TextGenerationStreamOutput]]:",
+        ") -> Union[str, TextGenerationOutput, AsyncIterable[str], AsyncIterable[TextGenerationStreamOutput]]:",
+    )
+
+    return code
+
+
+def _adapt_chat_completion_to_async(code: str) -> str:
+    # Catch `aiohttp` error instead of `requests` error
+    code = code.replace(
+        "except HTTPError:",
+        "except _import_aiohttp().ClientResponseError:",
+    )
+
+    # Await text-generation call
+    code = code.replace(
+        "text_generation_output = self.text_generation(",
+        "text_generation_output = await self.text_generation(",
+    )
+
+    # Update return types: Iterable -> AsyncIterable
+    code = code.replace(
+        ") -> Iterable[ChatCompletionStreamOutput]:",
+        ") -> AsyncIterable[ChatCompletionStreamOutput]:",
+    )
+    code = code.replace(
+        ") -> Union[ChatCompletionOutput, Iterable[ChatCompletionStreamOutput]]:",
+        ") -> Union[ChatCompletionOutput, AsyncIterable[ChatCompletionStreamOutput]]:",
     )
 
     return code
@@ -351,10 +399,18 @@ def _update_examples_in_public_methods(code: str) -> str:
 
 
 def _use_async_streaming_util(code: str) -> str:
-    return code.replace(
+    code = code.replace(
         "_stream_text_generation_response",
         "_async_stream_text_generation_response",
     )
+    code = code.replace(
+        "_stream_chat_completion_response_from_text_generation",
+        "_async_stream_chat_completion_response_from_text_generation",
+    )
+    code = code.replace(
+        "_stream_chat_completion_response_from_bytes", "_async_stream_chat_completion_response_from_bytes"
+    )
+    return code
 
 
 def _adapt_get_model_status(code: str) -> str:


Test patch:
diff --git a/tests/cassettes/InferenceClientVCRTest.test_chat_completion_no_stream.yaml b/tests/cassettes/InferenceClientVCRTest.test_chat_completion_no_stream.yaml
new file mode 100644
index 00000000..444ac16c
--- /dev/null
+++ b/tests/cassettes/InferenceClientVCRTest.test_chat_completion_no_stream.yaml
@@ -0,0 +1,72 @@
+interactions:
+- request:
+    body: '{"model": "tgi", "messages": [{"role": "system", "content": "You are a
+      helpful assistant."}, {"role": "user", "content": "What is deep learning?"}],
+      "max_tokens": 20, "seed": null, "stop": null, "temperature": 1.0, "top_p": null,
+      "stream": false}'
+    headers:
+      Accept:
+      - '*/*'
+      Accept-Encoding:
+      - gzip, deflate, br
+      Connection:
+      - keep-alive
+      Content-Length:
+      - '246'
+      Content-Type:
+      - application/json
+      X-Amzn-Trace-Id:
+      - 720a26cf-f4ef-4894-b780-e9908abfdaba
+      user-agent:
+      - unknown/None; hf_hub/0.22.0.dev0; python/3.10.12; torch/2.2.0; tensorflow/2.15.0.post1;
+        fastcore/1.5.23
+    method: POST
+    uri: https://api-inference.huggingface.co/models/HuggingFaceH4/zephyr-7b-beta/v1/chat/completions
+  response:
+    body:
+      string: '{"id":"","object":"text_completion","created":1710439510,"model":"HuggingFaceH4/zephyr-7b-beta","system_fingerprint":"1.4.3-sha-e6bb3ff","choices":[{"index":0,"message":{"role":"assistant","content":"Deep
+        learning is a subfield of machine learning that focuses on training artificial
+        neural networks with multiple layers of"},"logprobs":null,"finish_reason":"length"}],"usage":{"prompt_tokens":47,"completion_tokens":20,"total_tokens":67}}'
+    headers:
+      Connection:
+      - keep-alive
+      Content-Type:
+      - application/json
+      Date:
+      - Thu, 14 Mar 2024 18:05:10 GMT
+      Transfer-Encoding:
+      - chunked
+      access-control-allow-credentials:
+      - 'true'
+      access-control-allow-origin:
+      - '*'
+      vary:
+      - origin, Origin, Access-Control-Request-Method, Access-Control-Request-Headers
+      x-compute-characters:
+      - '103'
+      x-compute-time:
+      - '0.71081148'
+      x-compute-type:
+      - 1-a10-g
+      x-generated-tokens:
+      - '20'
+      x-inference-time:
+      - '707'
+      x-prompt-tokens:
+      - '47'
+      x-queue-time:
+      - '3'
+      x-request-id:
+      - ZDNjmqhBd-p2kWw8sI2hB
+      x-sha:
+      - b70e0c9a2d9e14bd1e812d3c398e5f313e93b473
+      x-time-per-token:
+      - '35'
+      x-total-time:
+      - '710'
+      x-validation-time:
+      - '0'
+    status:
+      code: 200
+      message: OK
+version: 1
diff --git a/tests/cassettes/InferenceClientVCRTest.test_chat_completion_with_non_tgi.yaml b/tests/cassettes/InferenceClientVCRTest.test_chat_completion_with_non_tgi.yaml
new file mode 100644
index 00000000..17065c83
--- /dev/null
+++ b/tests/cassettes/InferenceClientVCRTest.test_chat_completion_with_non_tgi.yaml
@@ -0,0 +1,232 @@
+interactions:
+- request:
+    body: '{"model": "tgi", "messages": [{"role": "system", "content": "You are a
+      helpful assistant."}, {"role": "user", "content": "What is deep learning?"}],
+      "max_tokens": 20, "seed": null, "stop": null, "temperature": 1.0, "top_p": null,
+      "stream": false}'
+    headers:
+      Accept:
+      - '*/*'
+      Accept-Encoding:
+      - gzip, deflate, br
+      Connection:
+      - keep-alive
+      Content-Length:
+      - '246'
+      Content-Type:
+      - application/json
+      X-Amzn-Trace-Id:
+      - 77cd3bdd-15ea-497c-917b-ca099516edb0
+      user-agent:
+      - unknown/None; hf_hub/0.22.0.dev0; python/3.10.12; torch/2.2.0; tensorflow/2.15.0.post1;
+        fastcore/1.5.23
+    method: POST
+    uri: https://api-inference.huggingface.co/models/microsoft/DialoGPT-small/v1/chat/completions
+  response:
+    body:
+      string: '{"error":"unknown error","warnings":["There was an inference error:
+        unknown error: can only concatenate str (not \"dict\") to str"]}'
+    headers:
+      Connection:
+      - keep-alive
+      Content-Type:
+      - application/json
+      Date:
+      - Thu, 14 Mar 2024 18:15:53 GMT
+      Transfer-Encoding:
+      - chunked
+      access-control-allow-credentials:
+      - 'true'
+      access-control-expose-headers:
+      - x-compute-type, x-compute-time
+      server:
+      - uvicorn
+      vary:
+      - Origin, Access-Control-Request-Method, Access-Control-Request-Headers
+      x-compute-time:
+      - '0.002'
+      x-compute-type:
+      - cpu
+      x-request-id:
+      - zht7CE9kOneA0JFdf2xSO
+      x-sha:
+      - 49c537161a457d5256512f9d2d38a87d81ae0f0e
+    status:
+      code: 500
+      message: Internal Server Error
+- request:
+    body: null
+    headers:
+      Accept:
+      - '*/*'
+      Accept-Encoding:
+      - gzip, deflate, br
+      Connection:
+      - keep-alive
+      X-Amzn-Trace-Id:
+      - bafe8e99-e804-4b64-a5be-3b68351678b7
+      user-agent:
+      - unknown/None; hf_hub/0.22.0.dev0; python/3.10.12; torch/2.2.0; tensorflow/2.15.0.post1;
+        fastcore/1.5.23
+    method: GET
+    uri: https://huggingface.co/api/models/microsoft/DialoGPT-small
+  response:
+    body:
+      string: '{"_id":"621ffdc136468d709f17dec4","id":"microsoft/DialoGPT-small","modelId":"microsoft/DialoGPT-small","author":"microsoft","sha":"49c537161a457d5256512f9d2d38a87d81ae0f0e","lastModified":"2024-02-29T15:48:41.000Z","private":false,"disabled":false,"gated":false,"pipeline_tag":"text-generation","tags":["transformers","pytorch","tf","jax","safetensors","gpt2","text-generation","conversational","arxiv:1911.00536","license:mit","autotrain_compatible","endpoints_compatible","has_space","text-generation-inference","region:us"],"downloads":34593,"library_name":"transformers","widgetData":[{"text":"Hey
+        my name is Julien! How are you?"},{"text":"Hey my name is Thomas! How are
+        you?"},{"text":"Hey my name is Mariama! How are you?"},{"text":"Hey my name
+        is Clara! How are you?"},{"text":"Hey my name is Julien! How are you?"},{"text":"Hi."}],"likes":78,"model-index":null,"config":{"architectures":["GPT2LMHeadModel"],"model_type":"gpt2","tokenizer_config":{"bos_token":"
","chat_template":"{%
+        for message in messages %}{{ message.content }}{{ eos_token }}{% endfor %}","eos_token":"
","pad_token":null,"unk_token":"
"}},"cardData":{"thumbnail":"https://huggingface.co/front/thumbnails/dialogpt.png","tags":["conversational"],"license":"mit"},"transformersInfo":{"auto_model":"AutoModelForCausalLM","pipeline_tag":"text-generation","processor":"AutoTokenizer"},"spaces":["HuggingFaceH4/open_llm_leaderboard","gsaivinay/open_llm_leaderboard","GTBench/GTBench","felixz/open_llm_leaderboard","vasu0508/Meena_Chatbot","rushic24/DialoGPT-Covid-Help-Doctor","b1sheng/kg_llm_leaderboard_test","OPTML-Group/UnlearnCanvas-Benchmark","akhaliq/DialoGPT-small","mikeee/convbot","Docfile/open_llm_leaderboard","wop/test","wop/microsoft-DialoGPT-small","rodrigomasini/data_only_open_llm_leaderboard","Codermaker/microsoft-DialoGPT-small","hivemind-personalized-chat/chat-gradio","tomkr000/scottbotai","Teedossantos/microsoft-DialoGPT-small","itsarsile/microsoft-DialoGPT-small","overlordx/critic","EATHARD/chatbot","Youssefk/StoryAI","rabosh/just_deploy_experience","lilyling/Vietnam_QA","lilyling/Spain","lilyling/Italiano_QA","lilyling/Chinese_QA","wdsawdsawdadad/Chatbot_REQUIRES_OPENAI_KEY","iesir/microsoft-DialoGPT-small","fisehara/microsoft-DialoGPT-small","phenixreturn/microsoft-DialoGPT-small","hack3arma/microsoft-DialoGPT-small","rebornrulz/microsoft-DialoGPT-small","Daezi/microsoft-DialoGPT-small","pngwn/open_llm_leaderboard","pngwn/open_llm_leaderboard_two","freddyaboulton/open_llm_leaderboard_two_fix","choco9966/LeaderboardTest","Keyven/KiKi-GPT","choco9966/open-ko-llm-leaderboard","smothiki/open_llm_leaderboard","hmdsdt/microsoft-DialoGPT-small","Alfasign/Check","kevinklam/GuessWhom","Fideloub/microsoft-DialoGPT-small","JenitaChristopher/microsoft-DialoGPT-small","neubla/neubla-llm-evaluation-board","jawill/nlp_chatbot","junkim100/Self-Improving-Leaderboard"],"safetensors":{"parameters":{"F16":175620096},"total":175620096},"siblings":[{"rfilename":".gitattributes"},{"rfilename":"README.md"},{"rfilename":"config.json"},{"rfilename":"flax_model.msgpack"},{"rfilename":"generation_config.json"},{"rfilename":"generation_config_for_conversational.json"},{"rfilename":"merges.txt"},{"rfilename":"model.safetensors"},{"rfilename":"pytorch_model.bin"},{"rfilename":"tf_model.h5"},{"rfilename":"tokenizer_config.json"},{"rfilename":"vocab.json"}],"createdAt":"2022-03-02T23:29:05.000Z"}'
+    headers:
+      Access-Control-Allow-Origin:
+      - https://huggingface.co
+      Access-Control-Expose-Headers:
+      - X-Repo-Commit,X-Request-Id,X-Error-Code,X-Error-Message,ETag,Link,Accept-Ranges,Content-Range
+      Connection:
+      - keep-alive
+      Content-Length:
+      - '3428'
+      Content-Type:
+      - application/json; charset=utf-8
+      Date:
+      - Thu, 14 Mar 2024 18:15:53 GMT
+      ETag:
+      - W/"d64-atgwYMr1LyaFHkv0i0vXsUWWqTg"
+      Referrer-Policy:
+      - strict-origin-when-cross-origin
+      Vary:
+      - Origin
+      Via:
+      - 1.1 a355d8f903a0cf5525893c863fcdf216.cloudfront.net (CloudFront)
+      X-Amz-Cf-Id:
+      - A_blYo_2xR1JlGemvvnKlRCi34oDItnk2sqs88E4F_afAPXKOHZyJw==
+      X-Amz-Cf-Pop:
+      - CDG52-P4
+      X-Cache:
+      - Miss from cloudfront
+      X-Powered-By:
+      - huggingface-moon
+      X-Request-Id:
+      - Root=1-65f33ed9-56e731325d32b4f7757e6deb;bafe8e99-e804-4b64-a5be-3b68351678b7
+      cross-origin-opener-policy:
+      - same-origin
+    status:
+      code: 200
+      message: OK
+- request:
+    body: '{"inputs": "You are a helpful assistant.
What is deep learning?
",
+      "parameters": {"best_of": null, "decoder_input_details": false, "details": true,
+      "do_sample": false, "max_new_tokens": 20, "repetition_penalty": null, "return_full_text":
+      false, "seed": null, "stop": [], "temperature": 1.0, "top_k": null, "top_p":
+      null, "truncate": null, "typical_p": null, "watermark": false}, "stream": false}'
+    headers:
+      Accept:
+      - '*/*'
+      Accept-Encoding:
+      - gzip, deflate, br
+      Connection:
+      - keep-alive
+      Content-Length:
+      - '419'
+      Content-Type:
+      - application/json
+      X-Amzn-Trace-Id:
+      - bc827a1b-0d62-4fe9-aa55-0b12e26b2f35
+      user-agent:
+      - unknown/None; hf_hub/0.22.0.dev0; python/3.10.12; torch/2.2.0; tensorflow/2.15.0.post1;
+        fastcore/1.5.23
+    method: POST
+    uri: https://api-inference.huggingface.co/models/microsoft/DialoGPT-small
+  response:
+    body:
+      string: '{"error":"The following `model_kwargs` are not used by the model: [''decoder_input_details'',
+        ''watermark'', ''details'', ''stop''] (note: typos in the generate arguments
+        will also show up in this list)","warnings":["There was an inference error:
+        The following `model_kwargs` are not used by the model: [''decoder_input_details'',
+        ''watermark'', ''details'', ''stop''] (note: typos in the generate arguments
+        will also show up in this list)"]}'
+    headers:
+      Connection:
+      - keep-alive
+      Content-Type:
+      - application/json
+      Date:
+      - Thu, 14 Mar 2024 18:15:54 GMT
+      Transfer-Encoding:
+      - chunked
+      access-control-allow-credentials:
+      - 'true'
+      access-control-expose-headers:
+      - x-compute-type, x-compute-time
+      server:
+      - uvicorn
+      vary:
+      - Origin, Access-Control-Request-Method, Access-Control-Request-Headers
+      x-compute-time:
+      - '0.003'
+      x-compute-type:
+      - cpu
+      x-request-id:
+      - TUW19mGc6wCDceqt7KLQD
+      x-sha:
+      - 49c537161a457d5256512f9d2d38a87d81ae0f0e
+    status:
+      code: 400
+      message: Bad Request
+- request:
+    body: '{"inputs": "You are a helpful assistant.
What is deep learning?
",
+      "parameters": {"do_sample": false, "max_new_tokens": 20, "repetition_penalty":
+      null, "return_full_text": false, "seed": null, "temperature": 1.0, "top_k":
+      null, "top_p": null, "truncate": null, "typical_p": null}, "stream": false}'
+    headers:
+      Accept:
+      - '*/*'
+      Accept-Encoding:
+      - gzip, deflate, br
+      Connection:
+      - keep-alive
+      Content-Length:
+      - '321'
+      Content-Type:
+      - application/json
+      X-Amzn-Trace-Id:
+      - 266750af-5040-46ae-8097-184c267fc4e8
+      user-agent:
+      - unknown/None; hf_hub/0.22.0.dev0; python/3.10.12; torch/2.2.0; tensorflow/2.15.0.post1;
+        fastcore/1.5.23
+    method: POST
+    uri: https://api-inference.huggingface.co/models/microsoft/DialoGPT-small
+  response:
+    body:
+      string: '[{"generated_text":"Deep learning is a thing."}]'
+    headers:
+      Connection:
+      - keep-alive
+      Content-Type:
+      - application/json
+      Date:
+      - Thu, 14 Mar 2024 18:15:54 GMT
+      Transfer-Encoding:
+      - chunked
+      access-control-allow-credentials:
+      - 'true'
+      access-control-expose-headers:
+      - x-compute-type, x-compute-time
+      server:
+      - uvicorn
+      vary:
+      - Origin, Access-Control-Request-Method, Access-Control-Request-Headers
+      x-compute-characters:
+      - '76'
+      x-compute-time:
+      - '0.179'
+      x-compute-type:
+      - cpu
+      x-request-id:
+      - 2YOUZmqffbGsBiPEpgwBv
+      x-sha:
+      - 49c537161a457d5256512f9d2d38a87d81ae0f0e
+    status:
+      code: 200
+      message: OK
+version: 1
diff --git a/tests/cassettes/InferenceClientVCRTest.test_chat_completion_with_stream.yaml b/tests/cassettes/InferenceClientVCRTest.test_chat_completion_with_stream.yaml
new file mode 100644
index 00000000..2e6500ed
--- /dev/null
+++ b/tests/cassettes/InferenceClientVCRTest.test_chat_completion_with_stream.yaml
@@ -0,0 +1,139 @@
+interactions:
+- request:
+    body: '{"model": "tgi", "messages": [{"role": "system", "content": "You are a
+      helpful assistant."}, {"role": "user", "content": "What is deep learning?"}],
+      "max_tokens": 20, "seed": null, "stop": null, "temperature": 1.0, "top_p": null,
+      "stream": true}'
+    headers:
+      Accept:
+      - '*/*'
+      Accept-Encoding:
+      - gzip, deflate, br
+      Connection:
+      - keep-alive
+      Content-Length:
+      - '245'
+      Content-Type:
+      - application/json
+      X-Amzn-Trace-Id:
+      - d17fb8a1-42ab-43fd-b057-bf5e4446b26e
+      user-agent:
+      - unknown/None; hf_hub/0.22.0.dev0; python/3.10.12; torch/2.2.0; tensorflow/2.15.0.post1;
+        fastcore/1.5.23
+    method: POST
+    uri: https://api-inference.huggingface.co/models/HuggingFaceH4/zephyr-7b-beta/v1/chat/completions
+  response:
+    body:
+      string: 'data:{"id":"","object":"text_completion","created":1710439511,"model":"HuggingFaceH4/zephyr-7b-beta","system_fingerprint":"1.4.3-sha-e6bb3ff","choices":[{"index":0,"delta":{"role":"assistant","content":"Deep"},"logprobs":null,"finish_reason":null}]}
+
+
+        data:{"id":"","object":"text_completion","created":1710439511,"model":"HuggingFaceH4/zephyr-7b-beta","system_fingerprint":"1.4.3-sha-e6bb3ff","choices":[{"index":0,"delta":{"role":"assistant","content":"
+        learning"},"logprobs":null,"finish_reason":null}]}
+
+
+        data:{"id":"","object":"text_completion","created":1710439511,"model":"HuggingFaceH4/zephyr-7b-beta","system_fingerprint":"1.4.3-sha-e6bb3ff","choices":[{"index":0,"delta":{"role":"assistant","content":"
+        is"},"logprobs":null,"finish_reason":null}]}
+
+
+        data:{"id":"","object":"text_completion","created":1710439511,"model":"HuggingFaceH4/zephyr-7b-beta","system_fingerprint":"1.4.3-sha-e6bb3ff","choices":[{"index":0,"delta":{"role":"assistant","content":"
+        a"},"logprobs":null,"finish_reason":null}]}
+
+
+        data:{"id":"","object":"text_completion","created":1710439511,"model":"HuggingFaceH4/zephyr-7b-beta","system_fingerprint":"1.4.3-sha-e6bb3ff","choices":[{"index":0,"delta":{"role":"assistant","content":"
+        sub"},"logprobs":null,"finish_reason":null}]}
+
+
+        data:{"id":"","object":"text_completion","created":1710439511,"model":"HuggingFaceH4/zephyr-7b-beta","system_fingerprint":"1.4.3-sha-e6bb3ff","choices":[{"index":0,"delta":{"role":"assistant","content":"field"},"logprobs":null,"finish_reason":null}]}
+
+
+        data:{"id":"","object":"text_completion","created":1710439511,"model":"HuggingFaceH4/zephyr-7b-beta","system_fingerprint":"1.4.3-sha-e6bb3ff","choices":[{"index":0,"delta":{"role":"assistant","content":"
+        of"},"logprobs":null,"finish_reason":null}]}
+
+
+        data:{"id":"","object":"text_completion","created":1710439511,"model":"HuggingFaceH4/zephyr-7b-beta","system_fingerprint":"1.4.3-sha-e6bb3ff","choices":[{"index":0,"delta":{"role":"assistant","content":"
+        machine"},"logprobs":null,"finish_reason":null}]}
+
+
+        data:{"id":"","object":"text_completion","created":1710439511,"model":"HuggingFaceH4/zephyr-7b-beta","system_fingerprint":"1.4.3-sha-e6bb3ff","choices":[{"index":0,"delta":{"role":"assistant","content":"
+        learning"},"logprobs":null,"finish_reason":null}]}
+
+
+        data:{"id":"","object":"text_completion","created":1710439511,"model":"HuggingFaceH4/zephyr-7b-beta","system_fingerprint":"1.4.3-sha-e6bb3ff","choices":[{"index":0,"delta":{"role":"assistant","content":"
+        that"},"logprobs":null,"finish_reason":null}]}
+
+
+        data:{"id":"","object":"text_completion","created":1710439511,"model":"HuggingFaceH4/zephyr-7b-beta","system_fingerprint":"1.4.3-sha-e6bb3ff","choices":[{"index":0,"delta":{"role":"assistant","content":"
+        is"},"logprobs":null,"finish_reason":null}]}
+
+
+        data:{"id":"","object":"text_completion","created":1710439511,"model":"HuggingFaceH4/zephyr-7b-beta","system_fingerprint":"1.4.3-sha-e6bb3ff","choices":[{"index":0,"delta":{"role":"assistant","content":"
+        based"},"logprobs":null,"finish_reason":null}]}
+
+
+        data:{"id":"","object":"text_completion","created":1710439511,"model":"HuggingFaceH4/zephyr-7b-beta","system_fingerprint":"1.4.3-sha-e6bb3ff","choices":[{"index":0,"delta":{"role":"assistant","content":"
+        on"},"logprobs":null,"finish_reason":null}]}
+
+
+        data:{"id":"","object":"text_completion","created":1710439511,"model":"HuggingFaceH4/zephyr-7b-beta","system_fingerprint":"1.4.3-sha-e6bb3ff","choices":[{"index":0,"delta":{"role":"assistant","content":"
+        artificial"},"logprobs":null,"finish_reason":null}]}
+
+
+        data:{"id":"","object":"text_completion","created":1710439511,"model":"HuggingFaceH4/zephyr-7b-beta","system_fingerprint":"1.4.3-sha-e6bb3ff","choices":[{"index":0,"delta":{"role":"assistant","content":"
+        neural"},"logprobs":null,"finish_reason":null}]}
+
+
+        data:{"id":"","object":"text_completion","created":1710439511,"model":"HuggingFaceH4/zephyr-7b-beta","system_fingerprint":"1.4.3-sha-e6bb3ff","choices":[{"index":0,"delta":{"role":"assistant","content":"
+        networks"},"logprobs":null,"finish_reason":null}]}
+
+
+        data:{"id":"","object":"text_completion","created":1710439511,"model":"HuggingFaceH4/zephyr-7b-beta","system_fingerprint":"1.4.3-sha-e6bb3ff","choices":[{"index":0,"delta":{"role":"assistant","content":"
+        with"},"logprobs":null,"finish_reason":null}]}
+
+
+        data:{"id":"","object":"text_completion","created":1710439511,"model":"HuggingFaceH4/zephyr-7b-beta","system_fingerprint":"1.4.3-sha-e6bb3ff","choices":[{"index":0,"delta":{"role":"assistant","content":"
+        multiple"},"logprobs":null,"finish_reason":null}]}
+
+
+        data:{"id":"","object":"text_completion","created":1710439511,"model":"HuggingFaceH4/zephyr-7b-beta","system_fingerprint":"1.4.3-sha-e6bb3ff","choices":[{"index":0,"delta":{"role":"assistant","content":"
+        layers"},"logprobs":null,"finish_reason":null}]}
+
+
+        data:{"id":"","object":"text_completion","created":1710439511,"model":"HuggingFaceH4/zephyr-7b-beta","system_fingerprint":"1.4.3-sha-e6bb3ff","choices":[{"index":0,"delta":{"role":"assistant","content":"
+        to"},"logprobs":null,"finish_reason":null}]}
+
+
+        data:{"id":"","object":"text_completion","created":1710439511,"model":"HuggingFaceH4/zephyr-7b-beta","system_fingerprint":"1.4.3-sha-e6bb3ff","choices":[{"index":0,"delta":{},"logprobs":null,"finish_reason":"length"}]}
+
+
+        '
+    headers:
+      Connection:
+      - keep-alive
+      Content-Type:
+      - text/event-stream
+      Date:
+      - Thu, 14 Mar 2024 18:05:11 GMT
+      Transfer-Encoding:
+      - chunked
+      access-control-allow-credentials:
+      - 'true'
+      access-control-allow-origin:
+      - '*'
+      cache-control:
+      - no-cache
+      vary:
+      - origin, Origin, Access-Control-Request-Method, Access-Control-Request-Headers
+      x-accel-buffering:
+      - 'no'
+      x-compute-characters:
+      - '103'
+      x-compute-type:
+      - 1-a10-g
+      x-request-id:
+      - 1Zvgij4dCmqVWfGG_HNgk
+      x-sha:
+      - b70e0c9a2d9e14bd1e812d3c398e5f313e93b473
+    status:
+      code: 200
+      message: OK
+version: 1
diff --git a/tests/cassettes/test_async_chat_completion_no_stream.yaml b/tests/cassettes/test_async_chat_completion_no_stream.yaml
new file mode 100644
index 00000000..c2f36417
--- /dev/null
+++ b/tests/cassettes/test_async_chat_completion_no_stream.yaml
@@ -0,0 +1,57 @@
+interactions:
+- request:
+    body: null
+    headers:
+      user-agent:
+      - unknown/None; hf_hub/0.22.0.dev0; python/3.10.12; torch/2.2.0; tensorflow/2.15.0.post1;
+        fastcore/1.5.23
+    method: POST
+    uri: https://api-inference.huggingface.co/models/HuggingFaceH4/zephyr-7b-beta/v1/chat/completions
+  response:
+    body:
+      string: '{"id":"","object":"text_completion","created":1710439507,"model":"HuggingFaceH4/zephyr-7b-beta","system_fingerprint":"1.4.3-sha-e6bb3ff","choices":[{"index":0,"message":{"role":"assistant","content":"Deep
+        learning is a subfield of machine learning that"},"logprobs":null,"finish_reason":"length"}],"usage":{"prompt_tokens":47,"completion_tokens":10,"total_tokens":57}}'
+    headers:
+      Access-Control-Allow-Credentials:
+      - 'true'
+      Access-Control-Allow-Origin:
+      - '*'
+      Connection:
+      - keep-alive
+      Content-Type:
+      - application/json
+      Date:
+      - Thu, 14 Mar 2024 18:05:07 GMT
+      Transfer-Encoding:
+      - chunked
+      Vary:
+      - origin, Origin, Access-Control-Request-Method, Access-Control-Request-Headers
+      x-compute-characters:
+      - '103'
+      x-compute-time:
+      - '0.362352731'
+      x-compute-type:
+      - 1-a10-g
+      x-generated-tokens:
+      - '10'
+      x-inference-time:
+      - '357'
+      x-prompt-tokens:
+      - '47'
+      x-queue-time:
+      - '4'
+      x-request-id:
+      - q7E6J7NjMSoSFA3lPK44i
+      x-sha:
+      - b70e0c9a2d9e14bd1e812d3c398e5f313e93b473
+      x-time-per-token:
+      - '35'
+      x-total-time:
+      - '362'
+      x-validation-time:
+      - '0'
+    status:
+      code: 200
+      message: OK
+    url: https://api-inference.huggingface.co/models/HuggingFaceH4/zephyr-7b-beta/v1/chat/completions
+version: 1
diff --git a/tests/cassettes/test_async_chat_completion_not_tgi_no_stream.yaml b/tests/cassettes/test_async_chat_completion_not_tgi_no_stream.yaml
new file mode 100644
index 00000000..61a1a1bd
--- /dev/null
+++ b/tests/cassettes/test_async_chat_completion_not_tgi_no_stream.yaml
@@ -0,0 +1,189 @@
+interactions:
+- request:
+    body: null
+    headers:
+      user-agent:
+      - unknown/None; hf_hub/0.22.0.dev0; python/3.10.12; torch/2.2.0; tensorflow/2.15.0.post1;
+        fastcore/1.5.23
+    method: POST
+    uri: https://api-inference.huggingface.co/models/microsoft/DialoGPT-small/v1/chat/completions
+  response:
+    body:
+      string: '{"error":"unknown error","warnings":["There was an inference error:
+        unknown error: can only concatenate str (not \"dict\") to str"]}'
+    headers:
+      Access-Control-Allow-Credentials:
+      - 'true'
+      Access-Control-Expose-Headers:
+      - x-compute-type, x-compute-time
+      Connection:
+      - keep-alive
+      Content-Type:
+      - application/json
+      Date:
+      - Fri, 15 Mar 2024 10:20:28 GMT
+      Server:
+      - uvicorn
+      Transfer-Encoding:
+      - chunked
+      Vary:
+      - Origin, Access-Control-Request-Method, Access-Control-Request-Headers
+      x-compute-time:
+      - '0.002'
+      x-compute-type:
+      - cpu
+      x-request-id:
+      - e_GNvQmOIgRSusaT946Hj
+      x-sha:
+      - 49c537161a457d5256512f9d2d38a87d81ae0f0e
+    status:
+      code: 500
+      message: Internal Server Error
+    url: https://api-inference.huggingface.co/models/microsoft/DialoGPT-small/v1/chat/completions
+- request:
+    body: null
+    headers:
+      Accept:
+      - '*/*'
+      Accept-Encoding:
+      - gzip, deflate, br
+      Connection:
+      - keep-alive
+      X-Amzn-Trace-Id:
+      - 64f7a62f-dd29-4f75-a867-c178f7091cec
+      user-agent:
+      - unknown/None; hf_hub/0.22.0.dev0; python/3.10.12; torch/2.2.0; tensorflow/2.15.0.post1;
+        fastcore/1.5.23
+    method: GET
+    uri: https://huggingface.co/api/models/microsoft/DialoGPT-small
+  response:
+    body:
+      string: '{"_id":"621ffdc136468d709f17dec4","id":"microsoft/DialoGPT-small","modelId":"microsoft/DialoGPT-small","author":"microsoft","sha":"49c537161a457d5256512f9d2d38a87d81ae0f0e","lastModified":"2024-02-29T15:48:41.000Z","private":false,"disabled":false,"gated":false,"pipeline_tag":"text-generation","tags":["transformers","pytorch","tf","jax","safetensors","gpt2","text-generation","conversational","arxiv:1911.00536","license:mit","autotrain_compatible","endpoints_compatible","has_space","text-generation-inference","region:us"],"downloads":33570,"library_name":"transformers","widgetData":[{"text":"Hey
+        my name is Julien! How are you?"},{"text":"Hey my name is Thomas! How are
+        you?"},{"text":"Hey my name is Mariama! How are you?"},{"text":"Hey my name
+        is Clara! How are you?"},{"text":"Hey my name is Julien! How are you?"},{"text":"Hi."}],"likes":79,"model-index":null,"config":{"architectures":["GPT2LMHeadModel"],"model_type":"gpt2","tokenizer_config":{"bos_token":"
","chat_template":"{%
+        for message in messages %}{{ message.content }}{{ eos_token }}{% endfor %}","eos_token":"
","pad_token":null,"unk_token":"
"}},"cardData":{"thumbnail":"https://huggingface.co/front/thumbnails/dialogpt.png","tags":["conversational"],"license":"mit"},"transformersInfo":{"auto_model":"AutoModelForCausalLM","pipeline_tag":"text-generation","processor":"AutoTokenizer"},"spaces":["HuggingFaceH4/open_llm_leaderboard","gsaivinay/open_llm_leaderboard","GTBench/GTBench","felixz/open_llm_leaderboard","vasu0508/Meena_Chatbot","rushic24/DialoGPT-Covid-Help-Doctor","b1sheng/kg_llm_leaderboard_test","OPTML-Group/UnlearnCanvas-Benchmark","akhaliq/DialoGPT-small","mikeee/convbot","Docfile/open_llm_leaderboard","wop/test","wop/microsoft-DialoGPT-small","rodrigomasini/data_only_open_llm_leaderboard","Codermaker/microsoft-DialoGPT-small","hivemind-personalized-chat/chat-gradio","tomkr000/scottbotai","Teedossantos/microsoft-DialoGPT-small","itsarsile/microsoft-DialoGPT-small","overlordx/critic","EATHARD/chatbot","Youssefk/StoryAI","rabosh/just_deploy_experience","lilyling/Vietnam_QA","lilyling/Spain","lilyling/Italiano_QA","lilyling/Chinese_QA","wdsawdsawdadad/Chatbot_REQUIRES_OPENAI_KEY","iesir/microsoft-DialoGPT-small","fisehara/microsoft-DialoGPT-small","phenixreturn/microsoft-DialoGPT-small","hack3arma/microsoft-DialoGPT-small","rebornrulz/microsoft-DialoGPT-small","Daezi/microsoft-DialoGPT-small","pngwn/open_llm_leaderboard","pngwn/open_llm_leaderboard_two","freddyaboulton/open_llm_leaderboard_two_fix","choco9966/LeaderboardTest","Keyven/KiKi-GPT","choco9966/open-ko-llm-leaderboard","smothiki/open_llm_leaderboard","hmdsdt/microsoft-DialoGPT-small","Alfasign/Check","kevinklam/GuessWhom","Fideloub/microsoft-DialoGPT-small","JenitaChristopher/microsoft-DialoGPT-small","neubla/neubla-llm-evaluation-board","jawill/nlp_chatbot","junkim100/Self-Improving-Leaderboard"],"safetensors":{"parameters":{"F16":175620096},"total":175620096},"siblings":[{"rfilename":".gitattributes"},{"rfilename":"README.md"},{"rfilename":"config.json"},{"rfilename":"flax_model.msgpack"},{"rfilename":"generation_config.json"},{"rfilename":"generation_config_for_conversational.json"},{"rfilename":"merges.txt"},{"rfilename":"model.safetensors"},{"rfilename":"pytorch_model.bin"},{"rfilename":"tf_model.h5"},{"rfilename":"tokenizer_config.json"},{"rfilename":"vocab.json"}],"createdAt":"2022-03-02T23:29:05.000Z"}'
+    headers:
+      Access-Control-Allow-Origin:
+      - https://huggingface.co
+      Access-Control-Expose-Headers:
+      - X-Repo-Commit,X-Request-Id,X-Error-Code,X-Error-Message,ETag,Link,Accept-Ranges,Content-Range
+      Connection:
+      - keep-alive
+      Content-Length:
+      - '3428'
+      Content-Type:
+      - application/json; charset=utf-8
+      Date:
+      - Fri, 15 Mar 2024 10:20:29 GMT
+      ETag:
+      - W/"d64-QkhMKaEY+/4MIJGzLpoS4MYumYc"
+      Referrer-Policy:
+      - strict-origin-when-cross-origin
+      Vary:
+      - Origin
+      Via:
+      - 1.1 10138b7f7e9a868032a16788e533ba0e.cloudfront.net (CloudFront)
+      X-Amz-Cf-Id:
+      - L3_KSby-FamqJpVHycAHx2ERdGxh-eKzWso2-rUNLpw2IsnQsxfqDA==
+      X-Amz-Cf-Pop:
+      - CDG52-P4
+      X-Cache:
+      - Miss from cloudfront
+      X-Powered-By:
+      - huggingface-moon
+      X-Request-Id:
+      - Root=1-65f420ed-2a4cd6394ff6444f4576977a;64f7a62f-dd29-4f75-a867-c178f7091cec
+      cross-origin-opener-policy:
+      - same-origin
+    status:
+      code: 200
+      message: OK
+- request:
+    body: null
+    headers:
+      user-agent:
+      - unknown/None; hf_hub/0.22.0.dev0; python/3.10.12; torch/2.2.0; tensorflow/2.15.0.post1;
+        fastcore/1.5.23
+    method: POST
+    uri: https://api-inference.huggingface.co/models/microsoft/DialoGPT-small
+  response:
+    body:
+      string: '{"error":"The following `model_kwargs` are not used by the model: [''details'',
+        ''watermark'', ''stop'', ''decoder_input_details''] (note: typos in the generate
+        arguments will also show up in this list)","warnings":["There was an inference
+        error: The following `model_kwargs` are not used by the model: [''details'',
+        ''watermark'', ''stop'', ''decoder_input_details''] (note: typos in the generate
+        arguments will also show up in this list)"]}'
+    headers:
+      Access-Control-Allow-Credentials:
+      - 'true'
+      Access-Control-Expose-Headers:
+      - x-compute-type, x-compute-time
+      Connection:
+      - keep-alive
+      Content-Type:
+      - application/json
+      Date:
+      - Fri, 15 Mar 2024 10:20:29 GMT
+      Server:
+      - uvicorn
+      Transfer-Encoding:
+      - chunked
+      Vary:
+      - Origin, Access-Control-Request-Method, Access-Control-Request-Headers
+      x-compute-time:
+      - '0.003'
+      x-compute-type:
+      - cpu
+      x-request-id:
+      - t7qWwwa1TOJ96IXnQdrXv
+      x-sha:
+      - 49c537161a457d5256512f9d2d38a87d81ae0f0e
+    status:
+      code: 400
+      message: Bad Request
+    url: https://api-inference.huggingface.co/models/microsoft/DialoGPT-small
+- request:
+    body: null
+    headers:
+      user-agent:
+      - unknown/None; hf_hub/0.22.0.dev0; python/3.10.12; torch/2.2.0; tensorflow/2.15.0.post1;
+        fastcore/1.5.23
+    method: POST
+    uri: https://api-inference.huggingface.co/models/microsoft/DialoGPT-small
+  response:
+    body:
+      string: '[{"generated_text":"Deep learning is a thing."}]'
+    headers:
+      Access-Control-Allow-Credentials:
+      - 'true'
+      Access-Control-Expose-Headers:
+      - x-compute-type, x-compute-time
+      Connection:
+      - keep-alive
+      Content-Type:
+      - application/json
+      Date:
+      - Fri, 15 Mar 2024 10:20:30 GMT
+      Server:
+      - uvicorn
+      Transfer-Encoding:
+      - chunked
+      Vary:
+      - Origin, Access-Control-Request-Method, Access-Control-Request-Headers
+      x-compute-characters:
+      - '76'
+      x-compute-time:
+      - '0.169'
+      x-compute-type:
+      - cpu
+      x-request-id:
+      - oDjQSzomFaFX3l8he4yhw
+      x-sha:
+      - 49c537161a457d5256512f9d2d38a87d81ae0f0e
+    status:
+      code: 200
+      message: OK
+    url: https://api-inference.huggingface.co/models/microsoft/DialoGPT-small
+version: 1
diff --git a/tests/cassettes/test_async_chat_completion_with_stream.yaml b/tests/cassettes/test_async_chat_completion_with_stream.yaml
new file mode 100644
index 00000000..338fd422
--- /dev/null
+++ b/tests/cassettes/test_async_chat_completion_with_stream.yaml
@@ -0,0 +1,82 @@
+interactions:
+- request:
+    body: null
+    headers:
+      user-agent:
+      - unknown/None; hf_hub/0.22.0.dev0; python/3.10.12; torch/2.2.0; tensorflow/2.15.0.post1;
+        fastcore/1.5.23
+    method: POST
+    uri: https://api-inference.huggingface.co/models/HuggingFaceH4/zephyr-7b-beta/v1/chat/completions
+  response:
+    body:
+      string: 'data:{"id":"","object":"text_completion","created":1710439508,"model":"HuggingFaceH4/zephyr-7b-beta","system_fingerprint":"1.4.3-sha-e6bb3ff","choices":[{"index":0,"delta":{"role":"assistant","content":"Deep"},"logprobs":null,"finish_reason":null}]}
+
+
+        data:{"id":"","object":"text_completion","created":1710439508,"model":"HuggingFaceH4/zephyr-7b-beta","system_fingerprint":"1.4.3-sha-e6bb3ff","choices":[{"index":0,"delta":{"role":"assistant","content":"
+        Learning"},"logprobs":null,"finish_reason":null}]}
+
+
+        data:{"id":"","object":"text_completion","created":1710439508,"model":"HuggingFaceH4/zephyr-7b-beta","system_fingerprint":"1.4.3-sha-e6bb3ff","choices":[{"index":0,"delta":{"role":"assistant","content":"
+        is"},"logprobs":null,"finish_reason":null}]}
+
+
+        data:{"id":"","object":"text_completion","created":1710439508,"model":"HuggingFaceH4/zephyr-7b-beta","system_fingerprint":"1.4.3-sha-e6bb3ff","choices":[{"index":0,"delta":{"role":"assistant","content":"
+        a"},"logprobs":null,"finish_reason":null}]}
+
+
+        data:{"id":"","object":"text_completion","created":1710439508,"model":"HuggingFaceH4/zephyr-7b-beta","system_fingerprint":"1.4.3-sha-e6bb3ff","choices":[{"index":0,"delta":{"role":"assistant","content":"
+        sub"},"logprobs":null,"finish_reason":null}]}
+
+
+        data:{"id":"","object":"text_completion","created":1710439508,"model":"HuggingFaceH4/zephyr-7b-beta","system_fingerprint":"1.4.3-sha-e6bb3ff","choices":[{"index":0,"delta":{"role":"assistant","content":"field"},"logprobs":null,"finish_reason":null}]}
+
+
+        data:{"id":"","object":"text_completion","created":1710439508,"model":"HuggingFaceH4/zephyr-7b-beta","system_fingerprint":"1.4.3-sha-e6bb3ff","choices":[{"index":0,"delta":{"role":"assistant","content":"
+        of"},"logprobs":null,"finish_reason":null}]}
+
+
+        data:{"id":"","object":"text_completion","created":1710439508,"model":"HuggingFaceH4/zephyr-7b-beta","system_fingerprint":"1.4.3-sha-e6bb3ff","choices":[{"index":0,"delta":{"role":"assistant","content":"
+        Machine"},"logprobs":null,"finish_reason":null}]}
+
+
+        data:{"id":"","object":"text_completion","created":1710439508,"model":"HuggingFaceH4/zephyr-7b-beta","system_fingerprint":"1.4.3-sha-e6bb3ff","choices":[{"index":0,"delta":{"role":"assistant","content":"
+        Learning"},"logprobs":null,"finish_reason":null}]}
+
+
+        data:{"id":"","object":"text_completion","created":1710439508,"model":"HuggingFaceH4/zephyr-7b-beta","system_fingerprint":"1.4.3-sha-e6bb3ff","choices":[{"index":0,"delta":{"role":"assistant","content":"
+        that"},"logprobs":null,"finish_reason":null}]}
+
+        data:{"id":"","object":"text_completion","created":1710439508,"model":"HuggingFaceH4/zephyr-7b-beta","system_fingerprint":"1.4.3-sha-e6bb3ff","choices":[{"index":0,"delta":{},"logprobs":null,"finish_reason":"length"}]}
+        '
+    headers:
+      Access-Control-Allow-Credentials:
+      - 'true'
+      Access-Control-Allow-Origin:
+      - '*'
+      Cache-Control:
+      - no-cache
+      Connection:
+      - keep-alive
+      Content-Type:
+      - text/event-stream
+      Date:
+      - Thu, 14 Mar 2024 18:05:08 GMT
+      Transfer-Encoding:
+      - chunked
+      Vary:
+      - origin, Origin, Access-Control-Request-Method, Access-Control-Request-Headers
+      x-accel-buffering:
+      - 'no'
+      x-compute-characters:
+      - '103'
+      x-compute-type:
+      - 1-a10-g
+      x-request-id:
+      - idvh81inTm9FBUT-za5t7
+      x-sha:
+      - b70e0c9a2d9e14bd1e812d3c398e5f313e93b473
+    status:
+      code: 200
+      message: OK
+    url: https://api-inference.huggingface.co/models/HuggingFaceH4/zephyr-7b-beta/v1/chat/completions
+version: 1
diff --git a/tests/test_inference_async_client.py b/tests/test_inference_async_client.py
index 58f97b7f..c3416c8c 100644
--- a/tests/test_inference_async_client.py
+++ b/tests/test_inference_async_client.py
@@ -30,10 +30,21 @@ import pytest
 from aiohttp import ClientResponseError
 
 import huggingface_hub.inference._common
-from huggingface_hub import AsyncInferenceClient, InferenceClient, InferenceTimeoutError
+from huggingface_hub import (
+    AsyncInferenceClient,
+    ChatCompletionOutput,
+    ChatCompletionOutputChoice,
+    ChatCompletionOutputChoiceMessage,
+    ChatCompletionStreamOutput,
+    InferenceClient,
+    InferenceTimeoutError,
+    TextGenerationPrefillToken,
+)
+from huggingface_hub.inference._common import ValidationError as TextGenerationValidationError
 from huggingface_hub.inference._common import _is_tgi_server
-from huggingface_hub.inference._text_generation import FinishReason, InputToken
-from huggingface_hub.inference._text_generation import ValidationError as TextGenerationValidationError
+
+from .test_inference_client import CHAT_COMPLETE_NON_TGI_MODEL, CHAT_COMPLETION_MESSAGES, CHAT_COMPLETION_MODEL
+from .testing_utils import with_production_testing
 
 
 @pytest.fixture(autouse=True)
@@ -59,11 +70,11 @@ async def test_async_generate_with_details(tgi_client: AsyncInferenceClient) ->
     response = await tgi_client.text_generation("test", details=True, max_new_tokens=1, decoder_input_details=True)
 
     assert response.generated_text == ""
-    assert response.details.finish_reason == FinishReason.Length
+    assert response.details.finish_reason == "length"
     assert response.details.generated_tokens == 1
     assert response.details.seed is None
     assert len(response.details.prefill) == 1
-    assert response.details.prefill[0] == InputToken(id=0, text="<pad>", logprob=None)
+    assert response.details.prefill[0] == TextGenerationPrefillToken(id=0, text="<pad>", logprob=None)
     assert len(response.details.tokens) == 1
     assert response.details.tokens[0].id == 3
     assert response.details.tokens[0].text == " "
@@ -137,11 +148,76 @@ async def test_async_generate_stream_with_details(tgi_client: AsyncInferenceClie
     response = responses[0]
 
     assert response.generated_text == ""
-    assert response.details.finish_reason == FinishReason.Length
+    assert response.details.finish_reason == "length"
     assert response.details.generated_tokens == 1
     assert response.details.seed is None
 
 
+@pytest.mark.vcr
+@pytest.mark.asyncio
+async def test_async_chat_completion_no_stream() -> None:
+    async_client = AsyncInferenceClient(model=CHAT_COMPLETION_MODEL)
+    output = await async_client.chat_completion(CHAT_COMPLETION_MESSAGES, max_tokens=10)
+    assert isinstance(output.created, int)
+    assert output == ChatCompletionOutput(
+        choices=[
+            ChatCompletionOutputChoice(
+                finish_reason="length",
+                index=0,
+                message=ChatCompletionOutputChoiceMessage(
+                    content="Deep learning is a subfield of machine learning that",
+                    role="assistant",
+                ),
+            )
+        ],
+        created=output.created,
+    )
+
+
+@pytest.mark.vcr
+@pytest.mark.asyncio
+@with_production_testing
+async def test_async_chat_completion_not_tgi_no_stream() -> None:
+    async_client = AsyncInferenceClient(model=CHAT_COMPLETE_NON_TGI_MODEL)
+    output = await async_client.chat_completion(CHAT_COMPLETION_MESSAGES, max_tokens=10)
+    assert isinstance(output.created, int)
+    assert output == ChatCompletionOutput(
+        choices=[
+            ChatCompletionOutputChoice(
+                finish_reason="unk",  # Non-TGI => cannot know the finish reason
+                index=0,
+                message=ChatCompletionOutputChoiceMessage(
+                    content="Deep learning is a thing.",
+                    role="assistant",
+                ),
+            )
+        ],
+        created=output.created,
+    )
+
+
+@pytest.mark.vcr
+@pytest.mark.asyncio
+async def test_async_chat_completion_with_stream() -> None:
+    async_client = AsyncInferenceClient(model=CHAT_COMPLETION_MODEL)
+    output = await async_client.chat_completion(CHAT_COMPLETION_MESSAGES, max_tokens=10, stream=True)
+
+    all_items = [item async for item in output]
+    generated_text = ""
+    for item in all_items[:-1]:  # all but last item
+        assert isinstance(item, ChatCompletionStreamOutput)
+        assert len(item.choices) == 1
+        generated_text += item.choices[0].delta.content
+    last_item = all_items[-1]
+
+    assert generated_text == "Deep Learning is a subfield of Machine Learning that"
+
+    # Last item has a finish reason but no role/content delta
+    assert last_item.choices[0].finish_reason == "length"
+    assert last_item.choices[0].delta.role is None
+    assert last_item.choices[0].delta.content is None
+
+
 @pytest.mark.vcr
 @pytest.mark.asyncio
 async def test_async_sentence_similarity() -> None:
@@ -162,22 +238,21 @@ def test_sync_vs_async_signatures() -> None:
     async_client = AsyncInferenceClient()
 
     # Some methods have to be tested separately.
-    special_methods = ["post", "text_generation"]
+    special_methods = ["post", "text_generation", "chat_completion"]
 
     # Post: this is not automatically tested. No need to test its signature separately.
 
-    # Text-generation: return type changes from Iterable[...] to AsyncIterable[...] but input parameters are the same
-    sync_method = getattr(client, "text_generation")
-    assert not inspect.iscoroutinefunction(sync_method)
-    async_method = getattr(async_client, "text_generation")
-    assert inspect.iscoroutinefunction(async_method)
-
-    sync_sig = inspect.signature(sync_method)
-    async_sig = inspect.signature(async_method)
-    assert sync_sig.parameters == async_sig.parameters
-    assert sync_sig.return_annotation != async_sig.return_annotation
+    # text-generation/chat-completion: return type changes from Iterable[...] to AsyncIterable[...] but input parameters are the same
+    for name in ["text_generation", "chat_completion"]:
+        sync_method = getattr(client, name)
+        assert not inspect.iscoroutinefunction(sync_method)
+        async_method = getattr(async_client, name)
+        assert inspect.iscoroutinefunction(async_method)
 
-    [name for name in dir(client) if (not name.startswith("_")) and inspect.ismethod(getattr(client, name))]
+        sync_sig = inspect.signature(sync_method)
+        async_sig = inspect.signature(async_method)
+        assert sync_sig.parameters == async_sig.parameters
+        assert sync_sig.return_annotation != async_sig.return_annotation
 
     # Check that all methods are consistent between InferenceClient and AsyncInferenceClient
     for name in dir(client):
@@ -261,5 +336,6 @@ async def test_async_generate_timeout_error(monkeypatch: pytest.MonkeyPatch) ->
 @pytest.mark.asyncio
 async def test_unprocessable_entity_error() -> None:
     with pytest.raises(ClientResponseError) as error:
-        await AsyncInferenceClient().conversational("Hi, who are you?", model="HuggingFaceH4/zephyr-7b-alpha")
+        with pytest.warns(FutureWarning, match=".*'InferenceClient.conversational'.*"):
+            await AsyncInferenceClient().conversational("Hi, who are you?", model="HuggingFaceH4/zephyr-7b-alpha")
     assert "Make sure 'conversational' task is supported by the model." in error.value.message
diff --git a/tests/test_inference_client.py b/tests/test_inference_client.py
index ee36aeac..63789cdc 100644
--- a/tests/test_inference_client.py
+++ b/tests/test_inference_client.py
@@ -12,6 +12,7 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 import io
+import time
 import unittest
 from pathlib import Path
 from unittest.mock import MagicMock, patch
@@ -22,6 +23,10 @@ from PIL import Image
 
 from huggingface_hub import (
     AutomaticSpeechRecognitionOutput,
+    ChatCompletionOutput,
+    ChatCompletionOutputChoice,
+    ChatCompletionOutputChoiceMessage,
+    ChatCompletionStreamOutput,
     DocumentQuestionAnsweringOutputElement,
     FillMaskOutputElement,
     ImageClassificationOutputElement,
@@ -41,7 +46,7 @@ from huggingface_hub.constants import ALL_INFERENCE_API_FRAMEWORKS, MAIN_INFEREN
 from huggingface_hub.inference._client import _open_as_binary
 from huggingface_hub.utils import HfHubHTTPError, build_hf_headers
 
-from .testing_utils import with_production_testing
+from .testing_utils import expect_deprecation, with_production_testing
 
 
 # Avoid call to hf.co/api/models in VCRed tests
@@ -70,6 +75,13 @@ _RECOMMENDED_MODELS_FOR_VCR = {
     "zero-shot-image-classification": "openai/clip-vit-base-patch32",
 }
 
+CHAT_COMPLETION_MODEL = "HuggingFaceH4/zephyr-7b-beta"
+CHAT_COMPLETION_MESSAGES = [
+    {"role": "system", "content": "You are a helpful assistant."},
+    {"role": "user", "content": "What is deep learning?"},
+]
+CHAT_COMPLETE_NON_TGI_MODEL = "microsoft/DialoGPT-small"
+
 
 class InferenceClientTest(unittest.TestCase):
     @classmethod
@@ -127,6 +139,82 @@ class InferenceClientVCRTest(InferenceClientTest):
             chunks=None,
         )
 
+    def test_chat_completion_no_stream(self) -> None:
+        output = self.client.chat_completion(
+            messages=CHAT_COMPLETION_MESSAGES,
+            model=CHAT_COMPLETION_MODEL,
+            stream=False,
+        )
+        assert isinstance(output, ChatCompletionOutput)
+        assert output.created < time.time()
+        assert output.choices == [
+            ChatCompletionOutputChoice(
+                finish_reason="length",
+                index=0,
+                message=ChatCompletionOutputChoiceMessage(
+                    content="Deep learning is a subfield of machine learning that focuses on training artificial neural networks with multiple layers of",
+                    role="assistant",
+                ),
+            )
+        ]
+
+    def test_chat_completion_with_stream(self) -> None:
+        output = list(
+            self.client.chat_completion(
+                messages=CHAT_COMPLETION_MESSAGES,
+                model=CHAT_COMPLETION_MODEL,
+                stream=True,
+                max_tokens=20,
+            )
+        )
+
+        assert isinstance(output, list)
+        assert all(isinstance(item, ChatCompletionStreamOutput) for item in output)
+        created = output[0].created
+        assert all(item.created == created for item in output)  # all tokens share the same timestamp
+
+        # All items except the last one have a single choice with role/content delta
+        for item in output[:-1]:
+            assert len(item.choices) == 1
+            assert item.choices[0].finish_reason is None
+            assert item.choices[0].index == 0
+            assert item.choices[0].delta.role == "assistant"
+            assert item.choices[0].delta.content is not None
+
+        # Last item has a finish reason but no role/content delta
+        assert output[-1].choices[0].finish_reason == "length"
+        assert output[-1].choices[0].delta.role is None
+        assert output[-1].choices[0].delta.content is None
+
+        # Reconstruct generated text
+        generated_text = "".join(
+            item.choices[0].delta.content for item in output if item.choices[0].delta.content is not None
+        )
+        expected_text = "Deep learning is a subfield of machine learning that is based on artificial neural networks with multiple layers to"
+        assert generated_text == expected_text
+
+    def test_chat_completion_with_non_tgi(self) -> None:
+        output = self.client.chat_completion(
+            messages=CHAT_COMPLETION_MESSAGES,
+            model=CHAT_COMPLETE_NON_TGI_MODEL,
+            stream=False,
+            max_tokens=20,
+        )
+        assert output == ChatCompletionOutput(
+            choices=[
+                ChatCompletionOutputChoice(
+                    finish_reason="unk",  # <- specific to models served with transformers (not possible to get details)
+                    index=0,
+                    message=ChatCompletionOutputChoiceMessage(
+                        content="Deep learning is a thing.",
+                        role="assistant",
+                    ),
+                )
+            ],
+            created=output.created,
+        )
+
+    @expect_deprecation("InferenceClient.conversational")
     def test_conversational(self) -> None:
         output = self.client.conversational("Hi, who are you?")
         self.assertEqual(
@@ -453,6 +541,7 @@ class InferenceClientVCRTest(InferenceClientTest):
             self.assertIsInstance(item.label, str)
             self.assertIsInstance(item.score, float)
 
+    @expect_deprecation("InferenceClient.conversational")
     def test_unprocessable_entity_error(self) -> None:
         with self.assertRaisesRegex(HfHubHTTPError, "Make sure 'conversational' task is supported by the model."):
             self.client.conversational("Hi, who are you?", model="HuggingFaceH4/zephyr-7b-alpha")
diff --git a/tests/test_inference_text_generation.py b/tests/test_inference_text_generation.py
index 88e1afcd..76bf35ee 100644
--- a/tests/test_inference_text_generation.py
+++ b/tests/test_inference_text_generation.py
@@ -7,136 +7,21 @@ from typing import Dict
 from unittest.mock import MagicMock, patch
 
 import pytest
-from pydantic import ValidationError
 from requests import HTTPError
 
-from huggingface_hub import InferenceClient
-from huggingface_hub.inference._common import _NON_TGI_SERVERS
-from huggingface_hub.inference._text_generation import (
-    FinishReason,
+from huggingface_hub import InferenceClient, TextGenerationPrefillToken
+from huggingface_hub.inference._common import (
+    _NON_TGI_SERVERS,
     GenerationError,
     IncompleteGenerationError,
-    InputToken,
     OverloadedError,
-    TextGenerationParameters,
-    TextGenerationRequest,
-    TextGenerationStreamResponse,
     raise_text_generation_error,
 )
-from huggingface_hub.inference._text_generation import (
+from huggingface_hub.inference._common import (
     ValidationError as TextGenerationValidationError,
 )
 
 
-class TestTextGenerationTypes(unittest.TestCase):
-    def test_parameters_validation(self):
-        # Test best_of
-        TextGenerationParameters(best_of=1)
-        with self.assertRaises(ValidationError):
-            TextGenerationParameters(best_of=0)
-        with self.assertRaises(ValidationError):
-            TextGenerationParameters(best_of=-1)
-        TextGenerationParameters(best_of=2, do_sample=True)
-        with self.assertRaises(ValidationError):
-            TextGenerationParameters(best_of=2)
-        with self.assertRaises(ValidationError):
-            TextGenerationParameters(best_of=2, seed=1)
-
-        # Test repetition_penalty
-        TextGenerationParameters(repetition_penalty=1)
-        with self.assertRaises(ValidationError):
-            TextGenerationParameters(repetition_penalty=0)
-        with self.assertRaises(ValidationError):
-            TextGenerationParameters(repetition_penalty=-1)
-
-        # Test seed
-        TextGenerationParameters(seed=1)
-        with self.assertRaises(ValidationError):
-            TextGenerationParameters(seed=-1)
-
-        # Test temperature
-        TextGenerationParameters(temperature=1)
-        with self.assertRaises(ValidationError):
-            TextGenerationParameters(temperature=0)
-        with self.assertRaises(ValidationError):
-            TextGenerationParameters(temperature=-1)
-
-        # Test top_k
-        TextGenerationParameters(top_k=1)
-        with self.assertRaises(ValidationError):
-            TextGenerationParameters(top_k=0)
-        with self.assertRaises(ValidationError):
-            TextGenerationParameters(top_k=-1)
-
-        # Test top_p
-        TextGenerationParameters(top_p=0.5)
-        with self.assertRaises(ValidationError):
-            TextGenerationParameters(top_p=0)
-        with self.assertRaises(ValidationError):
-            TextGenerationParameters(top_p=-1)
-        with self.assertRaises(ValidationError):
-            TextGenerationParameters(top_p=1)
-
-        # Test truncate
-        TextGenerationParameters(truncate=1)
-        with self.assertRaises(ValidationError):
-            TextGenerationParameters(truncate=0)
-        with self.assertRaises(ValidationError):
-            TextGenerationParameters(truncate=-1)
-
-        # Test typical_p
-        TextGenerationParameters(typical_p=0.5)
-        with self.assertRaises(ValidationError):
-            TextGenerationParameters(typical_p=0)
-        with self.assertRaises(ValidationError):
-            TextGenerationParameters(typical_p=-1)
-        with self.assertRaises(ValidationError):
-            TextGenerationParameters(typical_p=1)
-
-    def test_request_validation(self):
-        TextGenerationRequest(inputs="test")
-
-        with self.assertRaises(ValidationError):
-            TextGenerationRequest(inputs="")
-
-        TextGenerationRequest(inputs="test", stream=True)
-        TextGenerationRequest(inputs="test", parameters=TextGenerationParameters(best_of=2, do_sample=True))
-
-        with self.assertRaises(ValidationError):
-            TextGenerationRequest(
-                inputs="test", parameters=TextGenerationParameters(best_of=2, do_sample=True), stream=True
-            )
-
-    def test_streaming_response_validation(self):
-        """
-        Regression test for #2005.
-
-        See https://github.com/huggingface/huggingface_hub/issues/2005
-        """
-        json_payload_latest = {
-            "index": 97,
-            "token": {"id": 264, "text": " a", "logprob": -0.0003259182, "special": False},
-            "generated_text": None,
-            "details": None,
-        }
-        TextGenerationStreamResponse(**json_payload_latest)
-
-        json_payload_older_had_no_index = {
-            "token": {"id": 264, "text": " a", "logprob": -0.0003259182, "special": False},
-            "generated_text": None,
-            "details": None,
-        }
-        TextGenerationStreamResponse(**json_payload_older_had_no_index)
-
-        json_payload_without_required_field_token = {
-            "index": 97,
-            "generated_text": None,
-            "details": None,
-        }
-        with self.assertRaises((ValidationError, TypeError)):
-            TextGenerationStreamResponse(**json_payload_without_required_field_token)
-
-
 class TestTextGenerationErrors(unittest.TestCase):
     def test_generation_error(self):
         error = _mocked_error({"error_type": "generation", "error": "test"})
@@ -183,11 +68,11 @@ class TestTextGenerationClientVCR(unittest.TestCase):
         response = self.client.text_generation("test", details=True, max_new_tokens=1, decoder_input_details=True)
 
         assert response.generated_text == ""
-        assert response.details.finish_reason == FinishReason.Length
+        assert response.details.finish_reason == "length"
         assert response.details.generated_tokens == 1
         assert response.details.seed is None
         assert len(response.details.prefill) == 1
-        assert response.details.prefill[0] == InputToken(id=0, text="<pad>", logprob=None)
+        assert response.details.prefill[0] == TextGenerationPrefillToken(id=0, text="<pad>", logprob=None)
         assert len(response.details.tokens) == 1
         assert response.details.tokens[0].id == 3
         assert response.details.tokens[0].text == " "
@@ -216,7 +101,7 @@ class TestTextGenerationClientVCR(unittest.TestCase):
         response = responses[0]
 
         assert response.generated_text == ""
-        assert response.details.finish_reason == FinishReason.Length
+        assert response.details.finish_reason == "length"
         assert response.details.generated_tokens == 1
         assert response.details.seed is None
 
@@ -229,7 +114,7 @@ class TestTextGenerationClientVCR(unittest.TestCase):
         response = responses[0]
 
         assert response.generated_text == ""
-        assert response.details.finish_reason == FinishReason.Length
+        assert response.details.finish_reason == "length"
         assert response.details.generated_tokens == 1
         assert response.details.seed is None