Repository: huggingface/huggingface_hub
Pull request #2164 (2024-03-28 09:59:45)

🌐 [i18n-KO] Translated `guides/inference_endpoints.md` to Korean
<!-- PR의 제목은 "🌐 [i18n-KO] Translated `<your_file>.md` to Korean" 으로 부탁드립니다 -->
# What does this PR do?

Translated the `guides/inference_endpoints.md` file of the documentation to Korean.
Thank you in advance for your review. 

Part of https://github.com/huggingface/huggingface_hub/issues/1626
<!-- 메인 이슈에 기록이 남아요! 가짜연구소 리포를 사용해 연습하실때는 제거해주시면 감사하겠습니다! :smile: -->

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [x] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에만 가짜연구소 팀원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
Team PseudoLab, may you please review this PR? 
@jungnerd, @heuristicwave, @harheem, @cjfghk5697, @seoulsky-field, @nuatmochoi, @boyunJang, @SeungAhSon, @seoyoung-3060, @junejae

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

@jungnerd, @heuristicwave May you please review this PR? 

HuggingFaceDocBuilderDev: The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/huggingface_hub/pr_2164). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Patch:
diff --git a/.github/workflows/python-quality.yml b/.github/workflows/python-quality.yml
index 575b9f86..acb4e9a7 100644
--- a/.github/workflows/python-quality.yml
+++ b/.github/workflows/python-quality.yml
@@ -38,6 +38,7 @@ jobs:
       - run: .venv/bin/ruff check tests src contrib # linter
       - run: .venv/bin/ruff format --check tests src contrib # formatter
       - run: .venv/bin/python utils/check_contrib_list.py
+      - run: .venv/bin/python utils/check_inference_input_params.py
       - run: .venv/bin/python utils/check_static_imports.py
       - run: .venv/bin/python utils/generate_async_inference_client.py
       - run: .venv/bin/python utils/generate_inference_types.py
diff --git a/Makefile b/Makefile
index ad2a0520..140fd101 100644
--- a/Makefile
+++ b/Makefile
@@ -7,10 +7,11 @@ check_dirs := contrib src tests utils setup.py
 quality:
 	ruff check $(check_dirs)  # linter
 	ruff format --check $(check_dirs) # formatter
-	mypy src
+	python utils/check_inference_input_params.py
 	python utils/check_contrib_list.py
 	python utils/check_static_imports.py
 	python utils/generate_async_inference_client.py
+	mypy src
 
 style:
 	ruff check --fix $(check_dirs) # linter
diff --git a/docs/source/en/package_reference/inference_client.md b/docs/source/en/package_reference/inference_client.md
index f25aaeeb..2a6b37c6 100644
--- a/docs/source/en/package_reference/inference_client.md
+++ b/docs/source/en/package_reference/inference_client.md
@@ -37,18 +37,6 @@ pip install --upgrade huggingface_hub[inference]
 
 [[autodoc]] InferenceTimeoutError
 
-## Return types
-
-For most tasks, the return value has a built-in type (string, list, image...). Here is a list for the more complex types.
-
-### ConversationalOutputConversation
-
-[[autodoc]] huggingface_hub.inference._types.ConversationalOutputConversation
-
-### ConversationalOutput
-
-[[autodoc]] huggingface_hub.inference._types.ConversationalOutput
-
 ### ModelStatus
 
 [[autodoc]] huggingface_hub.inference._common.ModelStatus
diff --git a/docs/source/en/package_reference/inference_types.md b/docs/source/en/package_reference/inference_types.md
index 0cb67764..c0079d5e 100644
--- a/docs/source/en/package_reference/inference_types.md
+++ b/docs/source/en/package_reference/inference_types.md
@@ -53,13 +53,33 @@ This part of the lib is still under development and will be improved in future r
 
 [[autodoc]] huggingface_hub.ChatCompletionInput
 
+[[autodoc]] huggingface_hub.ChatCompletionInputFunctionDefinition
+
 [[autodoc]] huggingface_hub.ChatCompletionInputMessage
 
+[[autodoc]] huggingface_hub.ChatCompletionInputTool
+
+[[autodoc]] huggingface_hub.ChatCompletionInputToolCall
+
+[[autodoc]] huggingface_hub.ChatCompletionInputToolTypeClass
+
 [[autodoc]] huggingface_hub.ChatCompletionOutput
 
-[[autodoc]] huggingface_hub.ChatCompletionOutputChoice
+[[autodoc]] huggingface_hub.ChatCompletionOutputComplete
+
+[[autodoc]] huggingface_hub.ChatCompletionOutputFunctionDefinition
+
+[[autodoc]] huggingface_hub.ChatCompletionOutputLogprob
 
-[[autodoc]] huggingface_hub.ChatCompletionOutputChoiceMessage
+[[autodoc]] huggingface_hub.ChatCompletionOutputLogprobs
+
+[[autodoc]] huggingface_hub.ChatCompletionOutputMessage
+
+[[autodoc]] huggingface_hub.ChatCompletionOutputToolCall
+
+[[autodoc]] huggingface_hub.ChatCompletionOutputTopLogprob
+
+[[autodoc]] huggingface_hub.ChatCompletionOutputUsage
 
 [[autodoc]] huggingface_hub.ChatCompletionStreamOutput
 
@@ -67,6 +87,16 @@ This part of the lib is still under development and will be improved in future r
 
 [[autodoc]] huggingface_hub.ChatCompletionStreamOutputDelta
 
+[[autodoc]] huggingface_hub.ChatCompletionStreamOutputDeltaToolCall
+
+[[autodoc]] huggingface_hub.ChatCompletionStreamOutputFunction
+
+[[autodoc]] huggingface_hub.ChatCompletionStreamOutputLogprob
+
+[[autodoc]] huggingface_hub.ChatCompletionStreamOutputLogprobs
+
+[[autodoc]] huggingface_hub.ChatCompletionStreamOutputTopLogprob
+
 
 
 ## depth_estimation
@@ -225,21 +255,25 @@ This part of the lib is still under development and will be improved in future r
 
 [[autodoc]] huggingface_hub.TextGenerationInput
 
+[[autodoc]] huggingface_hub.TextGenerationInputGenerateParameters
+
+[[autodoc]] huggingface_hub.TextGenerationInputGrammarType
+
 [[autodoc]] huggingface_hub.TextGenerationOutput
 
+[[autodoc]] huggingface_hub.TextGenerationOutputBestOfSequence
+
 [[autodoc]] huggingface_hub.TextGenerationOutputDetails
 
-[[autodoc]] huggingface_hub.TextGenerationOutputSequenceDetails
+[[autodoc]] huggingface_hub.TextGenerationOutputPrefillToken
 
 [[autodoc]] huggingface_hub.TextGenerationOutputToken
 
-[[autodoc]] huggingface_hub.TextGenerationParameters
-
-[[autodoc]] huggingface_hub.TextGenerationPrefillToken
+[[autodoc]] huggingface_hub.TextGenerationStreamOutput
 
-[[autodoc]] huggingface_hub.TextGenerationStreamDetails
+[[autodoc]] huggingface_hub.TextGenerationStreamOutputStreamDetails
 
-[[autodoc]] huggingface_hub.TextGenerationStreamOutput
+[[autodoc]] huggingface_hub.TextGenerationStreamOutputToken
 
 
 
diff --git a/docs/source/ko/_toctree.yml b/docs/source/ko/_toctree.yml
index c6067b20..9198d0a4 100644
--- a/docs/source/ko/_toctree.yml
+++ b/docs/source/ko/_toctree.yml
@@ -24,6 +24,8 @@
       title: Hub에서 검색하기
     - local: guides/inference
       title: 추론
+    - local: guides/inference_endpoints
+      title: 추론 엔드포인트
     - local: guides/community
       title: 커뮤니티
     - local: in_translation
@@ -50,12 +52,12 @@
       title: (번역 중) Environment variables
     - local: package_reference/repository
       title: 로컬 및 온라인 리포지토리 관리
-    - local: in_translation
-      title: (번역 중) Hugging Face Hub API
+    - local: package_reference/hf_api
+      title: 허깅페이스 Hub API
     - local: package_reference/file_download
       title: 파일 다운로드하기
-    - local: in_translation
-      title: (번역 중) Mixins & serialization methods
+    - local: package_reference/mixins
+      title: 믹스인 & 직렬화 메소드
     - local: package_reference/inference_types
       title: 추론 타입
     - local: package_reference/inference_client
diff --git a/docs/source/ko/guides/inference_endpoints.md b/docs/source/ko/guides/inference_endpoints.md
new file mode 100644
index 00000000..ecd6dc27
--- /dev/null
+++ b/docs/source/ko/guides/inference_endpoints.md
@@ -0,0 +1,257 @@
+# 추론 엔드포인트[[inference-endpoints]]
+
+추론 엔드포인트는 Hugging Face가 관리하는 전용 및 자동 확장 인프라에 `transformers`, `sentence-transformers` 및 `diffusers` 모델을 쉽게 배포할 수 있는 안전한 프로덕션 솔루션을 제공합니다. 추론 엔드포인트는 [Hub](https://huggingface.co/models)의 모델로 구축됩니다.
+이 가이드에서는 `huggingface_hub`를 사용하여 프로그래밍 방식으로 추론 엔드포인트를 관리하는 방법을 배웁니다. 추론 엔드포인트 제품 자체에 대한 자세한 내용은 [공식 문서](https://huggingface.co/docs/inference-endpoints/index)를 참조하세요.
+
+이 가이드에서는 `huggingface_hub`가 올바르게 설치 및 로그인되어 있다고 가정합니다. 아직 그렇지 않은 경우 [빠른 시작 가이드](https://huggingface.co/docs/huggingface_hub/quick-start#quickstart)를 참조하세요. 추론 엔드포인트 API를 지원하는 최소 버전은 `v0.19.0`입니다.
+
+## 추론 엔드포인트 생성[[create-an-inference-endpoint]]
+
+첫 번째 단계는 [`create_inference_endpoint`]를 사용하여 추론 엔드포인트를 생성하는 것입니다:
+
+```py
+>>> from huggingface_hub import create_inference_endpoint
+
+>>> endpoint = create_inference_endpoint(
+...     "my-endpoint-name",
+...     repository="gpt2",
+...     framework="pytorch",
+...     task="text-generation",
+...     accelerator="cpu",
+...     vendor="aws",
+...     region="us-east-1",
+...     type="protected",
+...     instance_size="medium",
+...     instance_type="c6i"
+... )
+```
+
+예시에서는 `"my-endpoint-name"`라는 `protected` 추론 엔드포인트를 생성하여 `text-generation`을 위한 [gpt2](https://huggingface.co/gpt2)를 제공합니다. `protected` 추론 엔드포인트 API에 액세스하려면 토큰이 필요합니다. 또한 벤더, 지역, 액셀러레이터, 인스턴스 유형, 크기와 같은 하드웨어 요구 사항을 구성하기 위한 추가 정보를 제공해야 합니다. 사용 가능한 리소스 목록은 [여기](https://api.endpoints.huggingface.cloud/#/v2%3A%3Aprovider/list_vendors)에서 확인할 수 있습니다. 또한 [웹 인터페이스](https://ui.endpoints.huggingface.co/new)를 사용하여 편리하게 수동으로 추론 엔드포인트를 생성할 수 있습니다. 고급 설정 및 사용법에 대한 자세한 내용은 [이 가이드](https://huggingface.co/docs/inference-endpoints/guides/advanced)를 참조하세요.
+
+[`create_inference_endpoint`]에서 반환된 값은 [`InferenceEndpoint`] 개체입니다:
+
+```py
+>>> endpoint
+InferenceEndpoint(name='my-endpoint-name', namespace='Wauplin', repository='gpt2', status='pending', url=None)
+```
+
+이것은 엔드포인트에 대한 정보를 저장하는 데이터클래스입니다. `name`, `repository`, `status`, `task`, `created_at`, `updated_at` 등과 같은 중요한 속성에 접근할 수 있습니다. 필요한 경우 `endpoint.raw`를 통해 서버로부터의 원시 응답에도 접근할 수 있습니다.
+
+추론 엔드포인트가 생성되면 [개인 대시보드](https://ui.endpoints.huggingface.co/)에서 확인할 수 있습니다.
+
+![](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/huggingface_hub/inference_endpoints_created.png)
+
+#### 사용자 정의 이미지 사용[[using-a-custom-image]]
+
+기본적으로 추론 엔드포인트는 Hugging Face에서 제공하는 도커 이미지로 구축됩니다. 그러나 `custom_image` 매개변수를 사용하여 모든 도커 이미지를 지정할 수 있습니다. 일반적인 사용 사례는 [text-generation-inference](https://github.com/huggingface/text-generation-inference) 프레임워크를 사용하여 LLM을 실행하는 것입니다. 다음과 같이 수행할 수 있습니다:
+
+```python
+# TGI에서 Zephyr-7b-beta를 실행하는 추론 엔드포인트 시작하기
+>>> from huggingface_hub import create_inference_endpoint
+>>> endpoint = create_inference_endpoint(
+...     "aws-zephyr-7b-beta-0486",
+...     repository="HuggingFaceH4/zephyr-7b-beta",
+...     framework="pytorch",
+...     task="text-generation",
+...     accelerator="gpu",
+...     vendor="aws",
+...     region="us-east-1",
+...     type="protected",
+...     instance_size="medium",
+...     instance_type="g5.2xlarge",
+...     custom_image={
+...         "health_route": "/health",
+...         "env": {
+...             "MAX_BATCH_PREFILL_TOKENS": "2048",
+...             "MAX_INPUT_LENGTH": "1024",
+...             "MAX_TOTAL_TOKENS": "1512",
+...             "MODEL_ID": "/repository"
+...         },
+...         "url": "ghcr.io/huggingface/text-generation-inference:1.1.0",
+...     },
+... )
+```
+
+`custom_image`에 전달할 값은 도커 컨테이너의 URL과 이를 실행하기 위한 구성이 포함된 딕셔너리입니다. 자세한 내용은 [Swagger 문서](https://api.endpoints.huggingface.cloud/#/v2%3A%3Aendpoint/create_endpoint)를 참조하세요.
+
+### 기존 추론 엔드포인트 가져오기 또는 리스트 조회[[get-or-list-existing-inference-endpoints]]
+
+경우에 따라 이전에 생성한 추론 엔드포인트를 관리해야 할 수 있습니다. 이름을 알고 있는 경우 [`get_inference_endpoint`]를 사용하여 [`InferenceEndpoint`] 개체를 가져올 수 있습니다. 또는 [`list_inference_endpoints`]를 사용하여 모든 추론 엔드포인트 리스트를 검색할 수 있습니다. 두 메소드 모두 선택적 `namespace` 매개변수를 허용합니다. 속해 있는 조직의 `namespace`를 설정할 수 있습니다. 그렇지 않으면 기본적으로 사용자 이름이 사용됩니다.
+
+```py
+>>> from huggingface_hub import get_inference_endpoint, list_inference_endpoints
+
+# 엔드포인트 개체 가져오기
+>>> get_inference_endpoint("my-endpoint-name")
+InferenceEndpoint(name='my-endpoint-name', namespace='Wauplin', repository='gpt2', status='pending', url=None)
+
+# 조직의 모든 추론 엔드포인트 나열
+>>> list_inference_endpoints(namespace="huggingface")
+[InferenceEndpoint(name='aws-starchat-beta', namespace='huggingface', repository='HuggingFaceH4/starchat-beta', status='paused', url=None), ...]
+
+# 사용자가 속해있는 모든 조직의 엔드포인트 나열
+>>> list_inference_endpoints(namespace="*")
+[InferenceEndpoint(name='aws-starchat-beta', namespace='huggingface', repository='HuggingFaceH4/starchat-beta', status='paused', url=None), ...]
+```
+
+## 배포 상태 확인[[check-deployment-status]]
+
+이 가이드의 나머지 부분에서는 `endpoint`라는 이름의 [`InferenceEndpoint`] 객체를 가지고 있다고 가정합니다. 엔드포인트에 `status` 속성이 [`InferenceEndpointStatus`] 유형이라는 것을 알 수 있었습니다. 추론 엔드포인트가 배포되고 접근 가능하면 상태가 `"running"`이 되고 `url` 속성이 설정됩니다:
+
+```py
+>>> endpoint
+InferenceEndpoint(name='my-endpoint-name', namespace='Wauplin', repository='gpt2', status='running', url='https://jpj7k2q4j805b727.us-east-1.aws.endpoints.huggingface.cloud')
+```
+
+`추론 엔드포인트가 "running"` 상태에 도달하기 전에 일반적으로 `"initializing"` 또는 `"pending"` 단계를 거칩니다. [`~InferenceEndpoint.fetch`]를 실행하여 엔드포인트의 새로운 상태를 가져올 수 있습니다. [`InferenceEndpoint`]의 다른 메소드와 마찬가지로 이 메소드는 서버에 요청을 하며, `endpoint`의 내부 속성이 변경됩니다:
+
+```py
+>>> endpoint.fetch()
+InferenceEndpoint(name='my-endpoint-name', namespace='Wauplin', repository='gpt2', status='pending', url=None)
+```
+
+추론 엔드포인트가 실행될 때까지 기다리면서 상태를 가져오는 대신 [`~InferenceEndpoint.wait`]를 직접 호출할 수 있습니다. 이 헬퍼는 `timeout`과 `fetch_every` 매개변수를 입력으로 받아 (초 단위) 추론 엔드포인트가 배포될 때까지 스레드를 차단합니다. 기본값은 각각 `None`(제한 시간 없음)과 `5`초입니다.
+
+```py
+# 엔드포인트 보류
+>>> endpoint
+InferenceEndpoint(name='my-endpoint-name', namespace='Wauplin', repository='gpt2', status='pending', url=None)
+
+# 10초 대기 => InferenceEndpointTimeoutError 발생
+>>> endpoint.wait(timeout=10)
+    raise InferenceEndpointTimeoutError("Timeout while waiting for Inference Endpoint to be deployed.")
+huggingface_hub._inference_endpoints.InferenceEndpointTimeoutError: Timeout while waiting for Inference Endpoint to be deployed.
+
+# 추가 대기
+>>> endpoint.wait()
+InferenceEndpoint(name='my-endpoint-name', namespace='Wauplin', repository='gpt2', status='running', url='https://jpj7k2q4j805b727.us-east-1.aws.endpoints.huggingface.cloud')
+```
+
+`timeout`이 설정되어 있고 추론 엔드포인트를 불러오는 데 너무 오래 걸리면, [`InferenceEndpointTimeoutError`] 제한 시간 초과 오류가 발생합니다.
+
+## 추론 실행[[run-inference]]
+
+추론 엔드포인트가 실행되면, 마침내 추론을 실행할 수 있습니다!
+
+[`InferenceEndpoint`]에는 각각 [`InferenceClient`]와 [`AsyncInferenceClient`]를 반환하는 `client`와 `async_client` 속성이 있습니다.
+
+```py
+# 텍스트 생성 작업 실행:
+>>> endpoint.client.text_generation("I am")
+' not a fan of the idea of a "big-budget" movie. I think it\'s a'
+
+# 비동기 컨텍스트에서도 마찬가지로 실행:
+>>> await endpoint.async_client.text_generation("I am")
+```
+
+추론 엔드포인트가 실행 중이 아니면 [`InferenceEndpointError`] 오류가 발생합니다:
+
+```py
+>>> endpoint.client
+huggingface_hub._inference_endpoints.InferenceEndpointError: Cannot create a client for this Inference Endpoint as it is not yet deployed. Please wait for the Inference Endpoint to be deployed using `endpoint.wait()` and try again.
+```
+
+[`InferenceClient`]를 사용하는 방법에 대한 자세한 내용은 [추론 가이드](../guides/inference)를 참조하세요.
+
+## 라이프사이클 관리[[manage-lifecycle]]
+
+이제 추론 엔드포인트를 생성하고 추론을 실행하는 방법을 살펴보았으니, 라이프사이클을 관리하는 방법을 살펴봅시다.
+
+<Tip>
+
+이 섹션에서는 [`~InferenceEndpoint.pause`], [`~InferenceEndpoint.resume`], [`~InferenceEndpoint.scale_to_zero`], [`~InferenceEndpoint.update`] 및 [`~InferenceEndpoint.delete`] 등의 메소드를 살펴볼 것입니다. 모든 메소드는 편의를 위해 [`InferenceEndpoint`]에 추가된 별칭입니다. 원한다면 `HfApi`에 정의된 일반 메소드 [`pause_inference_endpoint`], [`resume_inference_endpoint`], [`scale_to_zero_inference_endpoint`], [`update_inference_endpoint`] 및 [`delete_inference_endpoint`]를 사용할 수도 있습니다.
+
+</Tip>
+
+### 일시 중지 또는 0으로 확장[[pause-or-scale-to-zero]]
+
+추론 엔드포인트를 사용하지 않을 때 비용을 절감하기 위해 [`~InferenceEndpoint.pause`]를 사용하여 일시 중지하거나 [`~InferenceEndpoint.scale_to_zero`]를 사용하여 0으로 스케일링할 수 있습니다.
+
+<Tip>
+
+*일시 중지* 또는 *0으로 스케일링*된 추론 엔드포인트는 비용이 들지 않습니다. 이 두 가지의 차이점은 *일시 중지* 엔드포인트는 [`~InferenceEndpoint.resume`]를 사용하여 명시적으로 *재개*해야 한다는 것입니다. 반대로 *0으로 스케일링*된 엔드포인트는 추론 호출이 있으면 추가 콜드 스타트 지연과 함께 자동으로 시작됩니다. 추론 엔드포인트는 일정 기간 비활성화된 후 자동으로 0으로 스케일링되도록 구성할 수도 있습니다.
+
+</Tip>
+
+```py
+# 엔드포인트 일시중지 및 재시작
+>>> endpoint.pause()
+InferenceEndpoint(name='my-endpoint-name', namespace='Wauplin', repository='gpt2', status='paused', url=None)
+>>> endpoint.resume()
+InferenceEndpoint(name='my-endpoint-name', namespace='Wauplin', repository='gpt2', status='pending', url=None)
+>>> endpoint.wait().client.text_generation(...)
+...
+
+# 0으로 스케일링
+>>> endpoint.scale_to_zero()
+InferenceEndpoint(name='my-endpoint-name', namespace='Wauplin', repository='gpt2', status='scaledToZero', url='https://jpj7k2q4j805b727.us-east-1.aws.endpoints.huggingface.cloud')
+# 엔드포인트는 'running'은 아니지만 URL�을 가지고 있으며 첫 번째 호출 시 다시 시작됩니다.
+```
+
+### 모델 또는 하드웨어 요구 사항 업데이트[[update-model-or-hardware-requirements]]
+
+경우에 따라 새로운 엔드포인트를 생성하지 않고 추론 엔드포인트를 업데이트하고 싶을 수 있습니다. 호스팅된 모델이나 모델 실행에 필요한 하드웨어 요구 사항을 업데이트할 수 있습니다. 이렇게 하려면 [`~InferenceEndpoint.update`]를 사용합니다:
+
+```py
+# 타겟 모델 변경
+>>> endpoint.update(repository="gpt2-large")
+InferenceEndpoint(name='my-endpoint-name', namespace='Wauplin', repository='gpt2-large', status='pending', url=None)
+
+# 복제본 갯수 업데이트
+>>> endpoint.update(min_replica=2, max_replica=6)
+InferenceEndpoint(name='my-endpoint-name', namespace='Wauplin', repository='gpt2-large', status='pending', url=None)
+
+# 더 큰 인스턴스로 업데이트
+>>> endpoint.update(accelerator="cpu", instance_size="large", instance_type="c6i")
+InferenceEndpoint(name='my-endpoint-name', namespace='Wauplin', repository='gpt2-large', status='pending', url=None)
+```
+
+### 엔드포인트 삭제[[delete-the-endpoint]]
+
+마지막으로 더 이상 추론 엔드포인트를 사용하지 않을 경우, [`~InferenceEndpoint.delete()`]를 호출하기만 하면 됩니다.
+
+<Tip warning={true}>
+
+이것은 돌이킬 수 없는 작업이며, 구성, 로그 및 사용 메트릭을 포함한 엔드포인트를 완전히 제거합니다. 삭제된 추론 엔드포인트는 복원할 수 없습니다.
+
+</Tip>
+
+## 엔드 투 엔드 예제[an-end-to-end-example]
+
+추론 엔드포인트의 일반적인 사용 사례는 한 번에 여러 개의 작업을 처리하여 인프라 비용을 제한하는 것입니다. 이 가이드에서 본 것을 사용하여 이 프로세스를 자동화할 수 있습니다:
+
+```py
+>>> import asyncio
+>>> from huggingface_hub import create_inference_endpoint
+
+# 엔드포인트 시작 + 초기화될 때까지 대기
+>>> endpoint = create_inference_endpoint(name="batch-endpoint",...).wait()
+
+# 추론 실행
+>>> client = endpoint.client
+>>> results = [client.text_generation(...) for job in jobs]
+
+# 비동기 추론 실행
+>>> async_client = endpoint.async_client
+>>> results = asyncio.gather(*[async_client.text_generation(...) for job in jobs])
+
+# 엔드포인트 중지
+>>> endpoint.pause()
+```
+
+또는 추론 엔드포인트가 이미 존재하고 일시 중지된 경우:
+
+```py
+>>> import asyncio
+>>> from huggingface_hub import get_inference_endpoint
+
+# 엔드포인트 가져오기 + 초기화될 때까지 대기
+>>> endpoint = get_inference_endpoint("batch-endpoint").resume().wait()
+
+# 추론 실행
+>>> async_client = endpoint.async_client
+>>> results = asyncio.gather(*[async_client.text_generation(...) for job in jobs])
+
+# 엔드포인트 중지
+>>> endpoint.pause()
+```
\ No newline at end of file
diff --git a/docs/source/ko/package_reference/hf_api.md b/docs/source/ko/package_reference/hf_api.md
new file mode 100644
index 00000000..862c7f59
--- /dev/null
+++ b/docs/source/ko/package_reference/hf_api.md
@@ -0,0 +1,126 @@
+<!--⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be
+rendered properly in your Markdown viewer.
+-->
+
+# HfApi Client[[hfapi-client]]
+
+아래는 허깅 페이스 Hub의 API를 위한 파이썬 래퍼인 `HfApi` 클래스에 대한 문서입니다.
+
+`HfApi`의 모든 메서드는 패키지의 루트에서 직접 접근할 수 있습니다. 두 접근 방식은 아래에서 자세히 설명합니다.
+
+루트 메서드를 사용하는 것이 더 간단하지만 [`HfApi`] 클래스를 사용하면 더 유연하게 사용할 수 있습니다.
+특히 모든 HTTP 호출에서 재사용할 토큰을 전달할 수 있습니다. 
+이 방식은 토큰이 머신에 유지되지 않기 때문에 `huggingface-cli login` 또는 [`login`]를 사용하는 방식과는 다르며,
+다른 엔드포인트를 제공하거나 사용자정의 에이전트를 구성할 수도 있습니다.
+
+```python
+from huggingface_hub import HfApi, list_models
+
+# 루트 메서드를 사용하세요.
+models = list_models()
+
+# 또는 HfApi client를 구성하세요.
+hf_api = HfApi(
+    endpoint="https://huggingface.co", # 비공개 Hub 엔드포인트를 지정할 수 있습니다.
+    token="hf_xxx", # 토큰은 머신에 유지되지 않습니다.
+)
+models = hf_api.list_models()
+```
+
+## HfApi[[huggingface_hub.HfApi]]
+
+[[autodoc]] HfApi
+
+[[autodoc]] plan_multi_commits
+
+## API Dataclasses[[api-dataclasses]]
+
+### AccessRequest[[huggingface_hub.hf_api.AccessRequest]]
+
+[[autodoc]] huggingface_hub.hf_api.AccessRequest
+
+### CommitInfo[[huggingface_hub.CommitInfo]]
+
+[[autodoc]] huggingface_hub.hf_api.CommitInfo
+
+### DatasetInfo[[huggingface_hub.hf_api.DatasetInfo]]
+
+[[autodoc]] huggingface_hub.hf_api.DatasetInfo
+
+### GitRefInfo[[huggingface_hub.GitRefInfo]]
+
+[[autodoc]] huggingface_hub.hf_api.GitRefInfo
+
+### GitCommitInfo[[huggingface_hub.GitCommitInfo]]
+
+[[autodoc]] huggingface_hub.hf_api.GitCommitInfo
+
+### GitRefs[[huggingface_hub.GitRefs]]
+
+[[autodoc]] huggingface_hub.hf_api.GitRefs
+
+### ModelInfo[[huggingface_hub.hf_api.ModelInfo]]
+
+[[autodoc]] huggingface_hub.hf_api.ModelInfo
+
+### RepoSibling[[huggingface_hub.hf_api.RepoSibling]]
+
+[[autodoc]] huggingface_hub.hf_api.RepoSibling
+
+### RepoFile[[huggingface_hub.hf_api.RepoFile]]
+
+[[autodoc]] huggingface_hub.hf_api.RepoFile
+
+### RepoUrl[[huggingface_hub.RepoUrl]]
+
+[[autodoc]] huggingface_hub.hf_api.RepoUrl
+
+### SafetensorsRepoMetadata[[huggingface_hub.utils.SafetensorsRepoMetadata]]
+
+[[autodoc]] huggingface_hub.utils.SafetensorsRepoMetadata
+
+### SafetensorsFileMetadata[[huggingface_hub.utils.SafetensorsFileMetadata]]
+
+[[autodoc]] huggingface_hub.utils.SafetensorsFileMetadata
+
+### SpaceInfo[[huggingface_hub.hf_api.SpaceInfo]]
+
+[[autodoc]] huggingface_hub.hf_api.SpaceInfo
+
+### TensorInfo[[huggingface_hub.utils.TensorInfo]]
+
+[[autodoc]] huggingface_hub.utils.TensorInfo
+
+### User[[huggingface_hub.User]]
+
+[[autodoc]] huggingface_hub.hf_api.User
+
+### UserLikes[[huggingface_hub.UserLikes]]
+
+[[autodoc]] huggingface_hub.hf_api.UserLikes
+
+## CommitOperation[[huggingface_hub.CommitOperationAdd]]
+
+[`CommitOperation`]에 지원되는 값은 다음과 같습니다:
+
+[[autodoc]] CommitOperationAdd
+
+[[autodoc]] CommitOperationDelete
+
+[[autodoc]] CommitOperationCopy
+
+## CommitScheduler[[huggingface_hub.CommitScheduler]]
+
+[[autodoc]] CommitScheduler
+
+## Search helpers[[search-helpers]]
+
+`huggingface_hub` 패키지에는 Hub에서 리포지토리를 필터링하는 데 도움되는 도구들이 포함되어 있습니다.
+
+### DatasetFilter[[huggingface_hub.DatasetFilter]]
+
+[[autodoc]] DatasetFilter
+
+### ModelFilter[[huggingface_hub.ModelFilter]]
+
+[[autodoc]] ModelFilter
diff --git a/docs/source/ko/package_reference/inference_types.md b/docs/source/ko/package_reference/inference_types.md
index 643b85ee..548249b5 100644
--- a/docs/source/ko/package_reference/inference_types.md
+++ b/docs/source/ko/package_reference/inference_types.md
@@ -9,7 +9,7 @@ rendered properly in your Markdown viewer.
 
 이 페이지에는 Hugging Face Hub에서 지원하는 타입(예: 데이터 클래스)이 나열되어 있습니다.
 각 작업은 JSON 스키마를 사용하여 지정되며, 이러한 스키마에 의해서 타입이 생성됩니다. 이때 Python 요구 사항으로 인해 일부 사용자 정의가 있을 수 있습니다.
- 
+
 각 작업의 JSON 스키마를 확인하려면 [@huggingface.js/tasks](https://github.com/huggingface/huggingface.js/tree/main/packages/tasks/src/tasks)를 확인하세요.
 
 라이브러리에서 이 부분은 아직 개발 중이며, 향후 릴리즈에서 개선될 예정입니다.
@@ -52,13 +52,33 @@ rendered properly in your Markdown viewer.
 
 [[autodoc]] huggingface_hub.ChatCompletionInput
 
+[[autodoc]] huggingface_hub.ChatCompletionInputFunctionDefinition
+
 [[autodoc]] huggingface_hub.ChatCompletionInputMessage
 
+[[autodoc]] huggingface_hub.ChatCompletionInputTool
+
+[[autodoc]] huggingface_hub.ChatCompletionInputToolCall
+
+[[autodoc]] huggingface_hub.ChatCompletionInputToolTypeClass
+
 [[autodoc]] huggingface_hub.ChatCompletionOutput
 
-[[autodoc]] huggingface_hub.ChatCompletionOutputChoice
+[[autodoc]] huggingface_hub.ChatCompletionOutputComplete
+
+[[autodoc]] huggingface_hub.ChatCompletionOutputFunctionDefinition
+
+[[autodoc]] huggingface_hub.ChatCompletionOutputLogprob
+
+[[autodoc]] huggingface_hub.ChatCompletionOutputLogprobs
+
+[[autodoc]] huggingface_hub.ChatCompletionOutputMessage
+
+[[autodoc]] huggingface_hub.ChatCompletionOutputToolCall
+
+[[autodoc]] huggingface_hub.ChatCompletionOutputTopLogprob
 
-[[autodoc]] huggingface_hub.ChatCompletionOutputChoiceMessage
+[[autodoc]] huggingface_hub.ChatCompletionOutputUsage
 
 [[autodoc]] huggingface_hub.ChatCompletionStreamOutput
 
@@ -66,6 +86,16 @@ rendered properly in your Markdown viewer.
 
 [[autodoc]] huggingface_hub.ChatCompletionStreamOutputDelta
 
+[[autodoc]] huggingface_hub.ChatCompletionStreamOutputDeltaToolCall
+
+[[autodoc]] huggingface_hub.ChatCompletionStreamOutputFunction
+
+[[autodoc]] huggingface_hub.ChatCompletionStreamOutputLogprob
+
+[[autodoc]] huggingface_hub.ChatCompletionStreamOutputLogprobs
+
+[[autodoc]] huggingface_hub.ChatCompletionStreamOutputTopLogprob
+
 
 
 ## depth_estimation[[huggingface_hub.DepthEstimationInput]]
@@ -224,21 +254,25 @@ rendered properly in your Markdown viewer.
 
 [[autodoc]] huggingface_hub.TextGenerationInput
 
+[[autodoc]] huggingface_hub.TextGenerationInputGenerateParameters
+
+[[autodoc]] huggingface_hub.TextGenerationInputGrammarType
+
 [[autodoc]] huggingface_hub.TextGenerationOutput
 
+[[autodoc]] huggingface_hub.TextGenerationOutputBestOfSequence
+
 [[autodoc]] huggingface_hub.TextGenerationOutputDetails
 
-[[autodoc]] huggingface_hub.TextGenerationOutputSequenceDetails
+[[autodoc]] huggingface_hub.TextGenerationOutputPrefillToken
 
 [[autodoc]] huggingface_hub.TextGenerationOutputToken
 
-[[autodoc]] huggingface_hub.TextGenerationParameters
-
-[[autodoc]] huggingface_hub.TextGenerationPrefillToken
+[[autodoc]] huggingface_hub.TextGenerationStreamOutput
 
-[[autodoc]] huggingface_hub.TextGenerationStreamDetails
+[[autodoc]] huggingface_hub.TextGenerationStreamOutputStreamDetails
 
-[[autodoc]] huggingface_hub.TextGenerationStreamOutput
+[[autodoc]] huggingface_hub.TextGenerationStreamOutputToken
 
 
 
diff --git a/docs/source/ko/package_reference/mixins.md b/docs/source/ko/package_reference/mixins.md
new file mode 100644
index 00000000..53aaf930
--- /dev/null
+++ b/docs/source/ko/package_reference/mixins.md
@@ -0,0 +1,37 @@
+<!--⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be
+rendered properly in your Markdown viewer.
+-->
+
+# 믹스인 & 직렬화 메소드[[mixins--serialization-methods]]
+
+## 믹스인[[mixins]]
+
+`huggingface_hub` 라이브러리는 객체에 함수들의 업로드 및 다운로드 기능을 손쉽게 제공하기 위해서, 부모 클래스로 사용될 수 있는 다양한 믹스인을 제공합니다.
+ML 프레임워크를 Hub와 통합하는 방법은 [통합 가이드](../guides/integrations)를 통해 배울 수 있습니다.
+
+### 제네릭[[huggingface_hub.ModelHubMixin]]
+
+[[autodoc]] ModelHubMixin
+    - all
+    - _save_pretrained
+    - _from_pretrained
+
+### PyTorch[[huggingface_hub.PyTorchModelHubMixin]]
+
+[[autodoc]] PyTorchModelHubMixin
+
+### Keras[[huggingface_hub.KerasModelHubMixin]]
+
+[[autodoc]] KerasModelHubMixin
+
+[[autodoc]] from_pretrained_keras
+
+[[autodoc]] push_to_hub_keras
+
+[[autodoc]] save_pretrained_keras
+
+### Fastai[[huggingface_hub.from_pretrained_fastai]]
+
+[[autodoc]] from_pretrained_fastai
+
+[[autodoc]] push_to_hub_fastai
diff --git a/src/huggingface_hub/__init__.py b/src/huggingface_hub/__init__.py
index 991ac71d..a6508ae3 100644
--- a/src/huggingface_hub/__init__.py
+++ b/src/huggingface_hub/__init__.py
@@ -270,13 +270,28 @@ _SUBMOD_ATTRS = {
         "AutomaticSpeechRecognitionOutputChunk",
         "AutomaticSpeechRecognitionParameters",
         "ChatCompletionInput",
+        "ChatCompletionInputFunctionDefinition",
         "ChatCompletionInputMessage",
+        "ChatCompletionInputTool",
+        "ChatCompletionInputToolCall",
+        "ChatCompletionInputToolTypeClass",
         "ChatCompletionOutput",
-        "ChatCompletionOutputChoice",
-        "ChatCompletionOutputChoiceMessage",
+        "ChatCompletionOutputComplete",
+        "ChatCompletionOutputFunctionDefinition",
+        "ChatCompletionOutputLogprob",
+        "ChatCompletionOutputLogprobs",
+        "ChatCompletionOutputMessage",
+        "ChatCompletionOutputToolCall",
+        "ChatCompletionOutputTopLogprob",
+        "ChatCompletionOutputUsage",
         "ChatCompletionStreamOutput",
         "ChatCompletionStreamOutputChoice",
         "ChatCompletionStreamOutputDelta",
+        "ChatCompletionStreamOutputDeltaToolCall",
+        "ChatCompletionStreamOutputFunction",
+        "ChatCompletionStreamOutputLogprob",
+        "ChatCompletionStreamOutputLogprobs",
+        "ChatCompletionStreamOutputTopLogprob",
         "DepthEstimationInput",
         "DepthEstimationOutput",
         "DocumentQuestionAnsweringInput",
@@ -324,14 +339,16 @@ _SUBMOD_ATTRS = {
         "TextClassificationOutputElement",
         "TextClassificationParameters",
         "TextGenerationInput",
+        "TextGenerationInputGenerateParameters",
+        "TextGenerationInputGrammarType",
         "TextGenerationOutput",
+        "TextGenerationOutputBestOfSequence",
         "TextGenerationOutputDetails",
-        "TextGenerationOutputSequenceDetails",
+        "TextGenerationOutputPrefillToken",
         "TextGenerationOutputToken",
-        "TextGenerationParameters",
-        "TextGenerationPrefillToken",
-        "TextGenerationStreamDetails",
         "TextGenerationStreamOutput",
+        "TextGenerationStreamOutputStreamDetails",
+        "TextGenerationStreamOutputToken",
         "TextToAudioGenerationParameters",
         "TextToAudioInput",
         "TextToAudioOutput",
@@ -731,13 +748,28 @@ if TYPE_CHECKING:  # pragma: no cover
         AutomaticSpeechRecognitionOutputChunk,  # noqa: F401
         AutomaticSpeechRecognitionParameters,  # noqa: F401
         ChatCompletionInput,  # noqa: F401
+        ChatCompletionInputFunctionDefinition,  # noqa: F401
         ChatCompletionInputMessage,  # noqa: F401
+        ChatCompletionInputTool,  # noqa: F401
+        ChatCompletionInputToolCall,  # noqa: F401
+        ChatCompletionInputToolTypeClass,  # noqa: F401
         ChatCompletionOutput,  # noqa: F401
-        ChatCompletionOutputChoice,  # noqa: F401
-        ChatCompletionOutputChoiceMessage,  # noqa: F401
+        ChatCompletionOutputComplete,  # noqa: F401
+        ChatCompletionOutputFunctionDefinition,  # noqa: F401
+        ChatCompletionOutputLogprob,  # noqa: F401
+        ChatCompletionOutputLogprobs,  # noqa: F401
+        ChatCompletionOutputMessage,  # noqa: F401
+        ChatCompletionOutputToolCall,  # noqa: F401
+        ChatCompletionOutputTopLogprob,  # noqa: F401
+        ChatCompletionOutputUsage,  # noqa: F401
         ChatCompletionStreamOutput,  # noqa: F401
         ChatCompletionStreamOutputChoice,  # noqa: F401
         ChatCompletionStreamOutputDelta,  # noqa: F401
+        ChatCompletionStreamOutputDeltaToolCall,  # noqa: F401
+        ChatCompletionStreamOutputFunction,  # noqa: F401
+        ChatCompletionStreamOutputLogprob,  # noqa: F401
+        ChatCompletionStreamOutputLogprobs,  # noqa: F401
+        ChatCompletionStreamOutputTopLogprob,  # noqa: F401
         DepthEstimationInput,  # noqa: F401
         DepthEstimationOutput,  # noqa: F401
         DocumentQuestionAnsweringInput,  # noqa: F401
@@ -785,14 +817,16 @@ if TYPE_CHECKING:  # pragma: no cover
         TextClassificationOutputElement,  # noqa: F401
         TextClassificationParameters,  # noqa: F401
         TextGenerationInput,  # noqa: F401
+        TextGenerationInputGenerateParameters,  # noqa: F401
+        TextGenerationInputGrammarType,  # noqa: F401
         TextGenerationOutput,  # noqa: F401
+        TextGenerationOutputBestOfSequence,  # noqa: F401
         TextGenerationOutputDetails,  # noqa: F401
-        TextGenerationOutputSequenceDetails,  # noqa: F401
+        TextGenerationOutputPrefillToken,  # noqa: F401
         TextGenerationOutputToken,  # noqa: F401
-        TextGenerationParameters,  # noqa: F401
-        TextGenerationPrefillToken,  # noqa: F401
-        TextGenerationStreamDetails,  # noqa: F401
         TextGenerationStreamOutput,  # noqa: F401
+        TextGenerationStreamOutputStreamDetails,  # noqa: F401
+        TextGenerationStreamOutputToken,  # noqa: F401
         TextToAudioGenerationParameters,  # noqa: F401
         TextToAudioInput,  # noqa: F401
         TextToAudioOutput,  # noqa: F401
diff --git a/src/huggingface_hub/hf_api.py b/src/huggingface_hub/hf_api.py
index 6b1c947c..7102b813 100644
--- a/src/huggingface_hub/hf_api.py
+++ b/src/huggingface_hub/hf_api.py
@@ -603,7 +603,7 @@ class ModelInfo:
             Is the repo gated.
             If so, whether there is manual or automatic approval.
         downloads (`int`):
-            Number of downloads of the model.
+            Number of downloads of the model over the last 30 days.
         likes (`int`):
             Number of likes of the model.
         library_name (`str`, *optional*):
@@ -756,7 +756,7 @@ class DatasetInfo:
             Is the repo gated.
             If so, whether there is manual or automatic approval.
         downloads (`int`):
-            Number of downloads of the dataset.
+            Number of downloads of the dataset over the last 30 days.
         likes (`int`):
             Number of likes of the dataset.
         tags (`List[str]`):
diff --git a/src/huggingface_hub/inference/_client.py b/src/huggingface_hub/inference/_client.py
index f091ede9..6b0b80e0 100644
--- a/src/huggingface_hub/inference/_client.py
+++ b/src/huggingface_hub/inference/_client.py
@@ -34,6 +34,7 @@
 # - Only the main parameters are publicly exposed. Power users can always read the docs for more options.
 import base64
 import logging
+import re
 import time
 import warnings
 from typing import (
@@ -63,12 +64,12 @@ from huggingface_hub.inference._common import (
     _bytes_to_image,
     _bytes_to_list,
     _fetch_recommended_models,
+    _get_unsupported_text_generation_kwargs,
     _import_numpy,
     _is_chat_completion_server,
-    _is_tgi_server,
     _open_as_binary,
     _set_as_non_chat_completion_server,
-    _set_as_non_tgi,
+    _set_unsupported_text_generation_kwargs,
     _stream_chat_completion_response_from_bytes,
     _stream_chat_completion_response_from_text_generation,
     _stream_text_generation_response,
@@ -78,9 +79,11 @@ from huggingface_hub.inference._generated.types import (
     AudioClassificationOutputElement,
     AudioToAudioOutputElement,
     AutomaticSpeechRecognitionOutput,
+    ChatCompletionInputTool,
+    ChatCompletionInputToolTypeClass,
     ChatCompletionOutput,
-    ChatCompletionOutputChoice,
-    ChatCompletionOutputChoiceMessage,
+    ChatCompletionOutputComplete,
+    ChatCompletionOutputMessage,
     ChatCompletionStreamOutput,
     DocumentQuestionAnsweringOutputElement,
     FillMaskOutputElement,
@@ -92,6 +95,7 @@ from huggingface_hub.inference._generated.types import (
     SummarizationOutput,
     TableQuestionAnsweringOutputElement,
     TextClassificationOutputElement,
+    TextGenerationInputGrammarType,
     TextGenerationOutput,
     TextGenerationStreamOutput,
     TokenClassificationOutputElement,
@@ -100,6 +104,7 @@ from huggingface_hub.inference._generated.types import (
     ZeroShotClassificationOutputElement,
     ZeroShotImageClassificationOutputElement,
 )
+from huggingface_hub.inference._generated.types.chat_completion import ChatCompletionInputToolTypeEnum
 from huggingface_hub.inference._templating import render_chat_prompt
 from huggingface_hub.inference._types import (
     ConversationalOutput,  # soon to be removed
@@ -119,6 +124,9 @@ if TYPE_CHECKING:
 logger = logging.getLogger(__name__)
 
 
+MODEL_KWARGS_NOT_USED_REGEX = re.compile(r"The following `model_kwargs` are not used by the model: \[(.*?)\]")
+
+
 class InferenceClient:
     """
     Initialize a new Inference Client.
@@ -416,10 +424,19 @@ class InferenceClient:
         *,
         model: Optional[str] = None,
         stream: Literal[False] = False,
-        max_tokens: int = 20,
+        frequency_penalty: Optional[float] = None,
+        logit_bias: Optional[List[float]] = None,
+        logprobs: Optional[bool] = None,
+        max_tokens: Optional[int] = None,
+        n: Optional[int] = None,
+        presence_penalty: Optional[float] = None,
         seed: Optional[int] = None,
-        stop: Optional[Union[List[str], str]] = None,
-        temperature: float = 1.0,
+        stop: Optional[List[str]] = None,
+        temperature: Optional[float] = None,
+        tool_choice: Optional[Union[ChatCompletionInputToolTypeClass, ChatCompletionInputToolTypeEnum]] = None,
+        tool_prompt: Optional[str] = None,
+        tools: Optional[List[ChatCompletionInputTool]] = None,
+        top_logprobs: Optional[int] = None,
         top_p: Optional[float] = None,
     ) -> ChatCompletionOutput: ...
 
@@ -430,10 +447,19 @@ class InferenceClient:
         *,
         model: Optional[str] = None,
         stream: Literal[True] = True,
-        max_tokens: int = 20,
+        frequency_penalty: Optional[float] = None,
+        logit_bias: Optional[List[float]] = None,
+        logprobs: Optional[bool] = None,
+        max_tokens: Optional[int] = None,
+        n: Optional[int] = None,
+        presence_penalty: Optional[float] = None,
         seed: Optional[int] = None,
-        stop: Optional[Union[List[str], str]] = None,
-        temperature: float = 1.0,
+        stop: Optional[List[str]] = None,
+        temperature: Optional[float] = None,
+        tool_choice: Optional[Union[ChatCompletionInputToolTypeClass, ChatCompletionInputToolTypeEnum]] = None,
+        tool_prompt: Optional[str] = None,
+        tools: Optional[List[ChatCompletionInputTool]] = None,
+        top_logprobs: Optional[int] = None,
         top_p: Optional[float] = None,
     ) -> Iterable[ChatCompletionStreamOutput]: ...
 
@@ -444,10 +470,19 @@ class InferenceClient:
         *,
         model: Optional[str] = None,
         stream: bool = False,
-        max_tokens: int = 20,
+        frequency_penalty: Optional[float] = None,
+        logit_bias: Optional[List[float]] = None,
+        logprobs: Optional[bool] = None,
+        max_tokens: Optional[int] = None,
+        n: Optional[int] = None,
+        presence_penalty: Optional[float] = None,
         seed: Optional[int] = None,
-        stop: Optional[Union[List[str], str]] = None,
-        temperature: float = 1.0,
+        stop: Optional[List[str]] = None,
+        temperature: Optional[float] = None,
+        tool_choice: Optional[Union[ChatCompletionInputToolTypeClass, ChatCompletionInputToolTypeEnum]] = None,
+        tool_prompt: Optional[str] = None,
+        tools: Optional[List[ChatCompletionInputTool]] = None,
+        top_logprobs: Optional[int] = None,
         top_p: Optional[float] = None,
     ) -> Union[ChatCompletionOutput, Iterable[ChatCompletionStreamOutput]]: ...
 
@@ -457,10 +492,20 @@ class InferenceClient:
         *,
         model: Optional[str] = None,
         stream: bool = False,
-        max_tokens: int = 20,
+        # Parameters from ChatCompletionInput (handled manually)
+        frequency_penalty: Optional[float] = None,
+        logit_bias: Optional[List[float]] = None,
+        logprobs: Optional[bool] = None,
+        max_tokens: Optional[int] = None,
+        n: Optional[int] = None,
+        presence_penalty: Optional[float] = None,
         seed: Optional[int] = None,
-        stop: Optional[Union[List[str], str]] = None,
-        temperature: float = 1.0,
+        stop: Optional[List[str]] = None,
+        temperature: Optional[float] = None,
+        tool_choice: Optional[Union[ChatCompletionInputToolTypeClass, ChatCompletionInputToolTypeEnum]] = None,
+        tool_prompt: Optional[str] = None,
+        tools: Optional[List[ChatCompletionInputTool]] = None,
+        top_logprobs: Optional[int] = None,
         top_p: Optional[float] = None,
     ) -> Union[ChatCompletionOutput, Iterable[ChatCompletionStreamOutput]]:
         """
@@ -483,27 +528,52 @@ class InferenceClient:
                 The model to use for chat-completion. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed
                 Inference Endpoint. If not provided, the default recommended model for chat-based text-generation will be used.
                 See https://huggingface.co/tasks/text-generation for more details.
-            frequency_penalty (`float`, optional):
+            frequency_penalty (`float`, *optional*):
                 Penalizes new tokens based on their existing frequency
                 in the text so far. Range: [-2.0, 2.0]. Defaults to 0.0.
-            max_tokens (`int`, optional):
+            logit_bias (`List[float]`, *optional*):
+                Modify the likelihood of specified tokens appearing in the completion. Accepts a JSON object that maps tokens
+                (specified by their token ID in the tokenizer) to an associated bias value from -100 to 100. Mathematically,
+                the bias is added to the logits generated by the model prior to sampling. The exact effect will vary per model,
+                but values between -1 and 1 should decrease or increase likelihood of selection; values like -100 or 100 should
+                result in a ban or exclusive selection of the relevant token. Defaults to None.
+            logprobs (`bool`, *optional*):
+                Whether to return log probabilities of the output tokens or not. If true, returns the log
+                probabilities of each output token returned in the content of message.
+            max_tokens (`int`, *optional*):
                 Maximum number of tokens allowed in the response. Defaults to 20.
-            seed (Optional[`int`], optional):
+            n (`int`, *optional*):
+                UNUSED.
+            presence_penalty (`float`, *optional*):
+                Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the
+                text so far, increasing the model's likelihood to talk about new topics.
+            seed (Optional[`int`], *optional*):
                 Seed for reproducible control flow. Defaults to None.
-            stop (Optional[`str`], optional):
+            stop (Optional[`str`], *optional*):
                 Up to four strings which trigger the end of the response.
                 Defaults to None.
-            stream (`bool`, optional):
+            stream (`bool`, *optional*):
                 Enable realtime streaming of responses. Defaults to False.
-            temperature (`float`, optional):
+            temperature (`float`, *optional*):
                 Controls randomness of the generations. Lower values ensure
                 less random completions. Range: [0, 2]. Defaults to 1.0.
-            top_p (`float`, optional):
+            top_logprobs (`int`, *optional*):
+                An integer between 0 and 5 specifying the number of most likely tokens to return at each token
+                position, each with an associated log probability. logprobs must be set to true if this parameter is
+                used.
+            top_p (`float`, *optional*):
                 Fraction of the most likely next words to sample from.
                 Must be between 0 and 1. Defaults to 1.0.
+            tool_choice ([`ChatCompletionInputToolTypeClass`] or [`ChatCompletionInputToolTypeEnum`], *optional*):
+                The tool to use for the completion. Defaults to "auto".
+            tool_prompt (`str`, *optional*):
+                A prompt to be appended before the tools.
+            tools (List of [`ChatCompletionInputTool`], *optional*):
+                A list of tools the model may call. Currently, only functions are supported as a tool. Use this to
+                provide a list of functions the model may generate JSON inputs for.
 
         Returns:
-            `Union[ChatCompletionOutput, Iterable[ChatCompletionStreamOutput]]`:
+            [`ChatCompletionOutput] or Iterable of [`ChatCompletionStreamOutput`]:
             Generated text returned from the server:
             - if `stream=False`, the generated text is returned as a [`ChatCompletionOutput`] (default).
             - if `stream=True`, the generated text is returned token by token as a sequence of [`ChatCompletionStreamOutput`].
@@ -515,18 +585,20 @@ class InferenceClient:
                 If the request fails with an HTTP error status code other than HTTP 503.
 
         Example:
+
         ```py
+        # Chat example
         >>> from huggingface_hub import InferenceClient
         >>> messages = [{"role": "user", "content": "What is the capital of France?"}]
         >>> client = InferenceClient("HuggingFaceH4/zephyr-7b-beta")
         >>> client.chat_completion(messages, max_tokens=100)
         ChatCompletionOutput(
             choices=[
-                ChatCompletionOutputChoice(
+                ChatCompletionOutputComplete(
                     finish_reason='eos_token',
                     index=0,
-                    message=ChatCompletionOutputChoiceMessage(
-                        content='The capital of France is Paris. The official name of the city is "Ville de Paris" (City of Paris) and the name of the country\'s governing body, which is located in Paris, is "La République française" (The French Republic). \nI hope that helps! Let me know if you need any further information.'
+                    message=ChatCompletionOutputMessage(
+                        content='The capital of France is Paris. The official name of the city is Ville de Paris (City of Paris) and the name of the country governing body, which is located in Paris, is La République française (The French Republic). \nI hope that helps! Let me know if you need any further information.'
                     )
                 )
             ],
@@ -539,7 +611,87 @@ class InferenceClient:
         ChatCompletionStreamOutput(choices=[ChatCompletionStreamOutputChoice(delta=ChatCompletionStreamOutputDelta(content=' capital', role='assistant'), index=0, finish_reason=None)], created=1710498504)
         (...)
         ChatCompletionStreamOutput(choices=[ChatCompletionStreamOutputChoice(delta=ChatCompletionStreamOutputDelta(content=' may', role='assistant'), index=0, finish_reason=None)], created=1710498504)
-        ChatCompletionStreamOutput(choices=[ChatCompletionStreamOutputChoice(delta=ChatCompletionStreamOutputDelta(content=None, role=None), index=0, finish_reason='length')], created=1710498504)
+
+        # Chat example with tools
+        >>> client = InferenceClient("meta-llama/Meta-Llama-3-70B-Instruct")
+        >>> messages = [
+        ...     {
+        ...         "role": "system",
+        ...         "content": "Don't make assumptions about what values to plug into functions. Ask for clarification if a user request is ambiguous.",
+        ...     },
+        ...     {
+        ...         "role": "user",
+        ...         "content": "What's the weather like the next 3 days in San Francisco, CA?",
+        ...     },
+        ... ]
+        >>> tools = [
+        ...     {
+        ...         "type": "function",
+        ...         "function": {
+        ...             "name": "get_current_weather",
+        ...             "description": "Get the current weather",
+        ...             "parameters": {
+        ...                 "type": "object",
+        ...                 "properties": {
+        ...                     "location": {
+        ...                         "type": "string",
+        ...                         "description": "The city and state, e.g. San Francisco, CA",
+        ...                     },
+        ...                     "format": {
+        ...                         "type": "string",
+        ...                         "enum": ["celsius", "fahrenheit"],
+        ...                         "description": "The temperature unit to use. Infer this from the users location.",
+        ...                     },
+        ...                 },
+        ...                 "required": ["location", "format"],
+        ...             },
+        ...         },
+        ...     },
+        ...     {
+        ...         "type": "function",
+        ...         "function": {
+        ...             "name": "get_n_day_weather_forecast",
+        ...             "description": "Get an N-day weather forecast",
+        ...             "parameters": {
+        ...                 "type": "object",
+        ...                 "properties": {
+        ...                     "location": {
+        ...                         "type": "string",
+        ...                         "description": "The city and state, e.g. San Francisco, CA",
+        ...                     },
+        ...                     "format": {
+        ...                         "type": "string",
+        ...                         "enum": ["celsius", "fahrenheit"],
+        ...                         "description": "The temperature unit to use. Infer this from the users location.",
+        ...                     },
+        ...                     "num_days": {
+        ...                         "type": "integer",
+        ...                         "description": "The number of days to forecast",
+        ...                     },
+        ...                 },
+        ...                 "required": ["location", "format", "num_days"],
+        ...             },
+        ...         },
+        ...     },
+        ... ]
+
+        >>> response = client.chat_completion(
+        ...     model="meta-llama/Meta-Llama-3-70B-Instruct",
+        ...     messages=messages,
+        ...     tools=tools,
+        ...     tool_choice="auto",
+        ...     max_tokens=500,
+        ... )
+        >>> response.choices[0].message.tool_calls[0].function
+        ChatCompletionOutputFunctionDefinition(
+            arguments={
+                'location': 'San Francisco, CA',
+                'format': 'fahrenheit',
+                'num_days': 3
+            },
+            name='get_n_day_weather_forecast',
+            description=None
+        )
         ```
         """
         # determine model
@@ -558,10 +710,19 @@ class InferenceClient:
                     json=dict(
                         model="tgi",  # random string
                         messages=messages,
+                        frequency_penalty=frequency_penalty,
+                        logit_bias=logit_bias,
+                        logprobs=logprobs,
                         max_tokens=max_tokens,
+                        n=n,
+                        presence_penalty=presence_penalty,
                         seed=seed,
                         stop=stop,
                         temperature=temperature,
+                        tool_choice=tool_choice,
+                        tool_prompt=tool_prompt,
+                        tools=tools,
+                        top_logprobs=top_logprobs,
                         top_p=top_p,
                         stream=stream,
                     ),
@@ -622,7 +783,6 @@ class InferenceClient:
         prompt = render_chat_prompt(model_id=model_id, token=self.token, messages=messages)
 
         # generate response
-        stop_sequences = [stop] if isinstance(stop, str) else stop
         text_generation_output = self.text_generation(
             prompt=prompt,
             details=True,
@@ -630,7 +790,7 @@ class InferenceClient:
             model=model,
             max_new_tokens=max_tokens,
             seed=seed,
-            stop_sequences=stop_sequences,
+            stop_sequences=stop,
             temperature=temperature,
             top_p=top_p,
         )
@@ -651,12 +811,17 @@ class InferenceClient:
             finish_reason = "unk"
 
         return ChatCompletionOutput(
+            id="dummy",
+            model="dummy",
+            object="dummy",
+            system_fingerprint="dummy",
+            usage=None,  # type: ignore # set to None as we don't want to provide fake information
             created=created,
             choices=[
-                ChatCompletionOutputChoice(
+                ChatCompletionOutputComplete(
                     finish_reason=finish_reason,  # type: ignore
                     index=0,
-                    message=ChatCompletionOutputChoiceMessage(
+                    message=ChatCompletionOutputMessage(
                         content=content,
                         role="assistant",
                     ),
@@ -1483,19 +1648,24 @@ class InferenceClient:
         details: Literal[False] = ...,
         stream: Literal[False] = ...,
         model: Optional[str] = None,
-        do_sample: bool = False,
-        max_new_tokens: int = 20,
+        # Parameters from `TextGenerationInputGenerateParameters` (maintained manually)
         best_of: Optional[int] = None,
+        decoder_input_details: Optional[bool] = None,
+        do_sample: Optional[bool] = False,  # Manual default value
+        frequency_penalty: Optional[float] = None,
+        grammar: Optional[TextGenerationInputGrammarType] = None,
+        max_new_tokens: Optional[int] = None,
         repetition_penalty: Optional[float] = None,
-        return_full_text: bool = False,
+        return_full_text: Optional[bool] = False,  # Manual default value
         seed: Optional[int] = None,
-        stop_sequences: Optional[List[str]] = None,
+        stop_sequences: Optional[List[str]] = None,  # Same as `stop`
         temperature: Optional[float] = None,
         top_k: Optional[int] = None,
+        top_n_tokens: Optional[int] = None,
         top_p: Optional[float] = None,
         truncate: Optional[int] = None,
         typical_p: Optional[float] = None,
-        watermark: bool = False,
+        watermark: Optional[bool] = None,
     ) -> str: ...
 
     @overload
@@ -1506,19 +1676,24 @@ class InferenceClient:
         details: Literal[True] = ...,
         stream: Literal[False] = ...,
         model: Optional[str] = None,
-        do_sample: bool = False,
-        max_new_tokens: int = 20,
+        # Parameters from `TextGenerationInputGenerateParameters` (maintained manually)
         best_of: Optional[int] = None,
+        decoder_input_details: Optional[bool] = None,
+        do_sample: Optional[bool] = False,  # Manual default value
+        frequency_penalty: Optional[float] = None,
+        grammar: Optional[TextGenerationInputGrammarType] = None,
+        max_new_tokens: Optional[int] = None,
         repetition_penalty: Optional[float] = None,
-        return_full_text: bool = False,
+        return_full_text: Optional[bool] = False,  # Manual default value
         seed: Optional[int] = None,
-        stop_sequences: Optional[List[str]] = None,
+        stop_sequences: Optional[List[str]] = None,  # Same as `stop`
         temperature: Optional[float] = None,
         top_k: Optional[int] = None,
+        top_n_tokens: Optional[int] = None,
         top_p: Optional[float] = None,
         truncate: Optional[int] = None,
         typical_p: Optional[float] = None,
-        watermark: bool = False,
+        watermark: Optional[bool] = None,
     ) -> TextGenerationOutput: ...
 
     @overload
@@ -1529,19 +1704,24 @@ class InferenceClient:
         details: Literal[False] = ...,
         stream: Literal[True] = ...,
         model: Optional[str] = None,
-        do_sample: bool = False,
-        max_new_tokens: int = 20,
+        # Parameters from `TextGenerationInputGenerateParameters` (maintained manually)
         best_of: Optional[int] = None,
+        decoder_input_details: Optional[bool] = None,
+        do_sample: Optional[bool] = False,  # Manual default value
+        frequency_penalty: Optional[float] = None,
+        grammar: Optional[TextGenerationInputGrammarType] = None,
+        max_new_tokens: Optional[int] = None,
         repetition_penalty: Optional[float] = None,
-        return_full_text: bool = False,
+        return_full_text: Optional[bool] = False,  # Manual default value
         seed: Optional[int] = None,
-        stop_sequences: Optional[List[str]] = None,
+        stop_sequences: Optional[List[str]] = None,  # Same as `stop`
         temperature: Optional[float] = None,
         top_k: Optional[int] = None,
+        top_n_tokens: Optional[int] = None,
         top_p: Optional[float] = None,
         truncate: Optional[int] = None,
         typical_p: Optional[float] = None,
-        watermark: bool = False,
+        watermark: Optional[bool] = None,
     ) -> Iterable[str]: ...
 
     @overload
@@ -1552,19 +1732,24 @@ class InferenceClient:
         details: Literal[True] = ...,
         stream: Literal[True] = ...,
         model: Optional[str] = None,
-        do_sample: bool = False,
-        max_new_tokens: int = 20,
+        # Parameters from `TextGenerationInputGenerateParameters` (maintained manually)
         best_of: Optional[int] = None,
+        decoder_input_details: Optional[bool] = None,
+        do_sample: Optional[bool] = False,  # Manual default value
+        frequency_penalty: Optional[float] = None,
+        grammar: Optional[TextGenerationInputGrammarType] = None,
+        max_new_tokens: Optional[int] = None,
         repetition_penalty: Optional[float] = None,
-        return_full_text: bool = False,
+        return_full_text: Optional[bool] = False,  # Manual default value
         seed: Optional[int] = None,
-        stop_sequences: Optional[List[str]] = None,
+        stop_sequences: Optional[List[str]] = None,  # Same as `stop`
         temperature: Optional[float] = None,
         top_k: Optional[int] = None,
+        top_n_tokens: Optional[int] = None,
         top_p: Optional[float] = None,
         truncate: Optional[int] = None,
         typical_p: Optional[float] = None,
-        watermark: bool = False,
+        watermark: Optional[bool] = None,
     ) -> Iterable[TextGenerationStreamOutput]: ...
 
     @overload
@@ -1575,19 +1760,24 @@ class InferenceClient:
         details: Literal[True] = ...,
         stream: bool = ...,
         model: Optional[str] = None,
-        do_sample: bool = False,
-        max_new_tokens: int = 20,
+        # Parameters from `TextGenerationInputGenerateParameters` (maintained manually)
         best_of: Optional[int] = None,
+        decoder_input_details: Optional[bool] = None,
+        do_sample: Optional[bool] = False,  # Manual default value
+        frequency_penalty: Optional[float] = None,
+        grammar: Optional[TextGenerationInputGrammarType] = None,
+        max_new_tokens: Optional[int] = None,
         repetition_penalty: Optional[float] = None,
-        return_full_text: bool = False,
+        return_full_text: Optional[bool] = False,  # Manual default value
         seed: Optional[int] = None,
-        stop_sequences: Optional[List[str]] = None,
+        stop_sequences: Optional[List[str]] = None,  # Same as `stop`
         temperature: Optional[float] = None,
         top_k: Optional[int] = None,
+        top_n_tokens: Optional[int] = None,
         top_p: Optional[float] = None,
         truncate: Optional[int] = None,
         typical_p: Optional[float] = None,
-        watermark: bool = False,
+        watermark: Optional[bool] = None,
     ) -> Union[TextGenerationOutput, Iterable[TextGenerationStreamOutput]]: ...
 
     def text_generation(
@@ -1597,20 +1787,24 @@ class InferenceClient:
         details: bool = False,
         stream: bool = False,
         model: Optional[str] = None,
-        do_sample: bool = False,
-        max_new_tokens: int = 20,
+        # Parameters from `TextGenerationInputGenerateParameters` (maintained manually)
         best_of: Optional[int] = None,
+        decoder_input_details: Optional[bool] = None,
+        do_sample: Optional[bool] = False,  # Manual default value
+        frequency_penalty: Optional[float] = None,
+        grammar: Optional[TextGenerationInputGrammarType] = None,
+        max_new_tokens: Optional[int] = None,
         repetition_penalty: Optional[float] = None,
-        return_full_text: bool = False,
+        return_full_text: Optional[bool] = False,  # Manual default value
         seed: Optional[int] = None,
-        stop_sequences: Optional[List[str]] = None,
+        stop_sequences: Optional[List[str]] = None,  # Same as `stop`
         temperature: Optional[float] = None,
         top_k: Optional[int] = None,
+        top_n_tokens: Optional[int] = None,
         top_p: Optional[float] = None,
         truncate: Optional[int] = None,
         typical_p: Optional[float] = None,
-        watermark: bool = False,
-        decoder_input_details: bool = False,
+        watermark: Optional[bool] = None,
     ) -> Union[str, TextGenerationOutput, Iterable[str], Iterable[TextGenerationStreamOutput]]:
         """
         Given a prompt, generate the following text.
@@ -1638,38 +1832,46 @@ class InferenceClient:
             model (`str`, *optional*):
                 The model to use for inference. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed
                 Inference Endpoint. This parameter overrides the model defined at the instance level. Defaults to None.
-            do_sample (`bool`):
+            best_of (`int`, *optional*):
+                Generate best_of sequences and return the one if the highest token logprobs.
+            decoder_input_details (`bool`, *optional*):
+                Return the decoder input token logprobs and ids. You must set `details=True` as well for it to be taken
+                into account. Defaults to `False`.
+            do_sample (`bool`, *optional*):
                 Activate logits sampling
-            max_new_tokens (`int`):
+            frequency_penalty (`float`, *optional*):
+                Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in
+                the text so far, decreasing the model's likelihood to repeat the same line verbatim.
+            grammar ([`TextGenerationInputGrammarType`], *optional*):
+                Grammar constraints. Can be either a JSONSchema or a regex.
+            max_new_tokens (`int`, *optional*):
                 Maximum number of generated tokens
-            best_of (`int`):
-                Generate best_of sequences and return the one if the highest token logprobs
-            repetition_penalty (`float`):
+            repetition_penalty (`float`, *optional*):
                 The parameter for repetition penalty. 1.0 means no penalty. See [this
                 paper](https://arxiv.org/pdf/1909.05858.pdf) for more details.
-            return_full_text (`bool`):
+            return_full_text (`bool`, *optional*):
                 Whether to prepend the prompt to the generated text
-            seed (`int`):
+            seed (`int`, *optional*):
                 Random sampling seed
-            stop_sequences (`List[str]`):
+            stop_sequences (`List[str]`, *optional*):
                 Stop generating tokens if a member of `stop_sequences` is generated
-            temperature (`float`):
+            temperature (`float`, *optional*):
                 The value used to module the logits distribution.
-            top_k (`int`):
+            top_n_tokens (`int`, *optional*):
+                Return information about the `top_n_tokens` most likely tokens at each generation step, instead of
+                just the sampled token.
+            top_k (`int`, *optional`):
                 The number of highest probability vocabulary tokens to keep for top-k-filtering.
-            top_p (`float`):
+            top_p (`float`, *optional`):
                 If set to < 1, only the smallest set of most probable tokens with probabilities that add up to `top_p` or
                 higher are kept for generation.
-            truncate (`int`):
-                Truncate inputs tokens to the given size
-            typical_p (`float`):
+            truncate (`int`, *optional`):
+                Truncate inputs tokens to the given size.
+            typical_p (`float`, *optional`):
                 Typical Decoding mass
                 See [Typical Decoding for Natural Language Generation](https://arxiv.org/abs/2202.00666) for more information
-            watermark (`bool`):
+            watermark (`bool`, *optional`):
                 Watermarking with [A Watermark for Large Language Models](https://arxiv.org/abs/2301.10226)
-            decoder_input_details (`bool`):
-                Return the decoder input token logprobs and ids. You must set `details=True` as well for it to be taken
-                into account. Defaults to `False`.
 
         Returns:
             `Union[str, TextGenerationOutput, Iterable[str], Iterable[TextGenerationStreamOutput]]`:
@@ -1721,10 +1923,10 @@ class InferenceClient:
                 generated_tokens=12,
                 seed=None,
                 prefill=[
-                    TextGenerationPrefillToken(id=487, text='The', logprob=None),
-                    TextGenerationPrefillToken(id=53789, text=' hugging', logprob=-13.171875),
+                    TextGenerationPrefillOutputToken(id=487, text='The', logprob=None),
+                    TextGenerationPrefillOutputToken(id=53789, text=' hugging', logprob=-13.171875),
                     (...)
-                    TextGenerationPrefillToken(id=204, text=' ', logprob=-7.0390625)
+                    TextGenerationPrefillOutputToken(id=204, text=' ', logprob=-7.0390625)
                 ],
                 tokens=[
                     TokenElement(id=1425, text='100', logprob=-1.0175781, special=False),
@@ -1758,8 +1960,35 @@ class InferenceClient:
             logprob=-0.5703125,
             special=False),
             generated_text='100% open source and built to be easy to use.',
-            details=TextGenerationStreamDetails(finish_reason='length', generated_tokens=12, seed=None)
+            details=TextGenerationStreamOutputStreamDetails(finish_reason='length', generated_tokens=12, seed=None)
         )
+
+        # Case 5: generate constrained output using grammar
+        >>> response = client.text_generation(
+        ...     prompt="I saw a puppy a cat and a raccoon during my bike ride in the park",
+        ...     model="HuggingFaceH4/zephyr-orpo-141b-A35b-v0.1",
+        ...     max_new_tokens=100,
+        ...     repetition_penalty=1.3,
+        ...     grammar={
+        ...         "type": "json",
+        ...         "value": {
+        ...             "properties": {
+        ...                 "location": {"type": "string"},
+        ...                 "activity": {"type": "string"},
+        ...                 "animals_seen": {"type": "integer", "minimum": 1, "maximum": 5},
+        ...                 "animals": {"type": "array", "items": {"type": "string"}},
+        ...             },
+        ...             "required": ["location", "activity", "animals_seen", "animals"],
+        ...         },
+        ...     },
+        ... )
+        >>> json.loads(response)
+        {
+            "activity": "bike riding",
+            "animals": ["puppy", "cat", "raccoon"],
+            "animals_seen": 3,
+            "location": "park"
+        }
         ```
         """
         if decoder_input_details and not details:
@@ -1770,41 +1999,48 @@ class InferenceClient:
             decoder_input_details = False
 
         # Build payload
+        parameters = {
+            "best_of": best_of,
+            "decoder_input_details": decoder_input_details,
+            "do_sample": do_sample,
+            "frequency_penalty": frequency_penalty,
+            "grammar": grammar,
+            "max_new_tokens": max_new_tokens,
+            "repetition_penalty": repetition_penalty,
+            "return_full_text": return_full_text,
+            "seed": seed,
+            "stop": stop_sequences if stop_sequences is not None else [],
+            "temperature": temperature,
+            "top_k": top_k,
+            "top_n_tokens": top_n_tokens,
+            "top_p": top_p,
+            "truncate": truncate,
+            "typical_p": typical_p,
+            "watermark": watermark,
+        }
+        parameters = {k: v for k, v in parameters.items() if v is not None}
         payload = {
             "inputs": prompt,
-            "parameters": {
-                "best_of": best_of,
-                "decoder_input_details": decoder_input_details,
-                "details": details,
-                "do_sample": do_sample,
-                "max_new_tokens": max_new_tokens,
-                "repetition_penalty": repetition_penalty,
-                "return_full_text": return_full_text,
-                "seed": seed,
-                "stop": stop_sequences if stop_sequences is not None else [],
-                "temperature": temperature,
-                "top_k": top_k,
-                "top_p": top_p,
-                "truncate": truncate,
-                "typical_p": typical_p,
-                "watermark": watermark,
-            },
+            "parameters": parameters,
             "stream": stream,
         }
 
         # Remove some parameters if not a TGI server
-        if not _is_tgi_server(model):
-            parameters: Dict = payload["parameters"]  # type: ignore [assignment]
+        unsupported_kwargs = _get_unsupported_text_generation_kwargs(model)
+        if len(unsupported_kwargs) > 0:
+            # The server does not support some parameters
+            # => means it is not a TGI server
+            # => remove unsupported parameters and warn the user
 
             ignored_parameters = []
-            for key in "watermark", "details", "decoder_input_details", "best_of", "stop", "return_full_text":
-                if parameters[key] is not None:
+            for key in unsupported_kwargs:
+                if parameters.get(key):
                     ignored_parameters.append(key)
-                del parameters[key]
+                parameters.pop(key, None)
             if len(ignored_parameters) > 0:
                 warnings.warn(
-                    "API endpoint/model for text-generation is not served via TGI. Ignoring parameters"
-                    f" {ignored_parameters}.",
+                    "API endpoint/model for text-generation is not served via TGI. Ignoring following parameters:"
+                    f" {', '.join(ignored_parameters)}.",
                     UserWarning,
                 )
             if details:
@@ -1824,27 +2060,32 @@ class InferenceClient:
         try:
             bytes_output = self.post(json=payload, model=model, task="text-generation", stream=stream)  # type: ignore
         except HTTPError as e:
-            if isinstance(e, BadRequestError) and "The following `model_kwargs` are not used by the model" in str(e):
-                _set_as_non_tgi(model)
+            match = MODEL_KWARGS_NOT_USED_REGEX.search(str(e))
+            if isinstance(e, BadRequestError) and match:
+                unused_params = [kwarg.strip("' ") for kwarg in match.group(1).split(",")]
+                _set_unsupported_text_generation_kwargs(model, unused_params)
                 return self.text_generation(  # type: ignore
                     prompt=prompt,
                     details=details,
                     stream=stream,
                     model=model,
+                    best_of=best_of,
+                    decoder_input_details=decoder_input_details,
                     do_sample=do_sample,
+                    frequency_penalty=frequency_penalty,
+                    grammar=grammar,
                     max_new_tokens=max_new_tokens,
-                    best_of=best_of,
                     repetition_penalty=repetition_penalty,
                     return_full_text=return_full_text,
                     seed=seed,
                     stop_sequences=stop_sequences,
                     temperature=temperature,
                     top_k=top_k,
+                    top_n_tokens=top_n_tokens,
                     top_p=top_p,
                     truncate=truncate,
                     typical_p=typical_p,
                     watermark=watermark,
-                    decoder_input_details=decoder_input_details,
                 )
             raise_text_generation_error(e)
 
diff --git a/src/huggingface_hub/inference/_common.py b/src/huggingface_hub/inference/_common.py
index 85ec476b..3640e8c9 100644
--- a/src/huggingface_hub/inference/_common.py
+++ b/src/huggingface_hub/inference/_common.py
@@ -331,6 +331,11 @@ def _format_chat_completion_stream_output_from_text_generation(
     if item.details is None:
         # new token generated => return delta
         return ChatCompletionStreamOutput(
+            # explicitly set 'dummy' values to reduce expectations from users
+            id="dummy",
+            model="dummy",
+            object="dummy",
+            system_fingerprint="dummy",
             choices=[
                 ChatCompletionStreamOutputChoice(
                     delta=ChatCompletionStreamOutputDelta(
@@ -346,9 +351,14 @@ def _format_chat_completion_stream_output_from_text_generation(
     else:
         # generation is completed => return finish reason
         return ChatCompletionStreamOutput(
+            # explicitly set 'dummy' values to reduce expectations from users
+            id="dummy",
+            model="dummy",
+            object="dummy",
+            system_fingerprint="dummy",
             choices=[
                 ChatCompletionStreamOutputChoice(
-                    delta=ChatCompletionStreamOutputDelta(),
+                    delta=ChatCompletionStreamOutputDelta(role="assistant"),
                     finish_reason=item.details.finish_reason,
                     index=0,
                 )
@@ -403,8 +413,8 @@ async def _async_yield_from(client: "ClientSession", response: "ClientResponse")
 # Both approaches have very similar APIs, but not exactly the same. What we do first in
 # the `text_generation` method is to assume the model is served via TGI. If we realize
 # it's not the case (i.e. we receive an HTTP 400 Bad Request), we fallback to the
-# default API with a warning message. We remember for each model if it's a TGI server
-# or not using `_NON_TGI_SERVERS` global variable.
+# default API with a warning message. When that's the case, We remember the unsupported
+# attributes for this model in the `_UNSUPPORTED_TEXT_GENERATION_KWARGS` global variable.
 #
 # In addition, TGI servers have a built-in API route for chat-completion, which is not
 # available on the default API. We use this route to provide a more consistent behavior
@@ -413,15 +423,15 @@ async def _async_yield_from(client: "ClientSession", response: "ClientResponse")
 # For more details, see https://github.com/huggingface/text-generation-inference and
 # https://huggingface.co/docs/api-inference/detailed_parameters#text-generation-task.
 
-_NON_TGI_SERVERS: Set[Optional[str]] = set()
+_UNSUPPORTED_TEXT_GENERATION_KWARGS: Dict[Optional[str], List[str]] = {}
 
 
-def _set_as_non_tgi(model: Optional[str]) -> None:
-    _NON_TGI_SERVERS.add(model)
+def _set_unsupported_text_generation_kwargs(model: Optional[str], unsupported_kwargs: List[str]) -> None:
+    _UNSUPPORTED_TEXT_GENERATION_KWARGS.setdefault(model, []).extend(unsupported_kwargs)
 
 
-def _is_tgi_server(model: Optional[str]) -> bool:
-    return model not in _NON_TGI_SERVERS
+def _get_unsupported_text_generation_kwargs(model: Optional[str]) -> List[str]:
+    return _UNSUPPORTED_TEXT_GENERATION_KWARGS.get(model, [])
 
 
 _NON_CHAT_COMPLETION_SERVER: Set[str] = set()
diff --git a/src/huggingface_hub/inference/_generated/_async_client.py b/src/huggingface_hub/inference/_generated/_async_client.py
index edc73835..abc7d98a 100644
--- a/src/huggingface_hub/inference/_generated/_async_client.py
+++ b/src/huggingface_hub/inference/_generated/_async_client.py
@@ -21,6 +21,7 @@
 import asyncio
 import base64
 import logging
+import re
 import time
 import warnings
 from typing import (
@@ -52,21 +53,23 @@ from huggingface_hub.inference._common import (
     _bytes_to_image,
     _bytes_to_list,
     _fetch_recommended_models,
+    _get_unsupported_text_generation_kwargs,
     _import_numpy,
     _is_chat_completion_server,
-    _is_tgi_server,
     _open_as_binary,
     _set_as_non_chat_completion_server,
-    _set_as_non_tgi,
+    _set_unsupported_text_generation_kwargs,
     raise_text_generation_error,
 )
 from huggingface_hub.inference._generated.types import (
     AudioClassificationOutputElement,
     AudioToAudioOutputElement,
     AutomaticSpeechRecognitionOutput,
+    ChatCompletionInputTool,
+    ChatCompletionInputToolTypeClass,
     ChatCompletionOutput,
-    ChatCompletionOutputChoice,
-    ChatCompletionOutputChoiceMessage,
+    ChatCompletionOutputComplete,
+    ChatCompletionOutputMessage,
     ChatCompletionStreamOutput,
     DocumentQuestionAnsweringOutputElement,
     FillMaskOutputElement,
@@ -78,6 +81,7 @@ from huggingface_hub.inference._generated.types import (
     SummarizationOutput,
     TableQuestionAnsweringOutputElement,
     TextClassificationOutputElement,
+    TextGenerationInputGrammarType,
     TextGenerationOutput,
     TextGenerationStreamOutput,
     TokenClassificationOutputElement,
@@ -86,6 +90,7 @@ from huggingface_hub.inference._generated.types import (
     ZeroShotClassificationOutputElement,
     ZeroShotImageClassificationOutputElement,
 )
+from huggingface_hub.inference._generated.types.chat_completion import ChatCompletionInputToolTypeEnum
 from huggingface_hub.inference._templating import render_chat_prompt
 from huggingface_hub.inference._types import (
     ConversationalOutput,  # soon to be removed
@@ -104,6 +109,9 @@ if TYPE_CHECKING:
 logger = logging.getLogger(__name__)
 
 
+MODEL_KWARGS_NOT_USED_REGEX = re.compile(r"The following `model_kwargs` are not used by the model: \[(.*?)\]")
+
+
 class AsyncInferenceClient:
     """
     Initialize a new Inference Client.
@@ -415,10 +423,19 @@ class AsyncInferenceClient:
         *,
         model: Optional[str] = None,
         stream: Literal[False] = False,
-        max_tokens: int = 20,
+        frequency_penalty: Optional[float] = None,
+        logit_bias: Optional[List[float]] = None,
+        logprobs: Optional[bool] = None,
+        max_tokens: Optional[int] = None,
+        n: Optional[int] = None,
+        presence_penalty: Optional[float] = None,
         seed: Optional[int] = None,
-        stop: Optional[Union[List[str], str]] = None,
-        temperature: float = 1.0,
+        stop: Optional[List[str]] = None,
+        temperature: Optional[float] = None,
+        tool_choice: Optional[Union[ChatCompletionInputToolTypeClass, ChatCompletionInputToolTypeEnum]] = None,
+        tool_prompt: Optional[str] = None,
+        tools: Optional[List[ChatCompletionInputTool]] = None,
+        top_logprobs: Optional[int] = None,
         top_p: Optional[float] = None,
     ) -> ChatCompletionOutput: ...
 
@@ -429,10 +446,19 @@ class AsyncInferenceClient:
         *,
         model: Optional[str] = None,
         stream: Literal[True] = True,
-        max_tokens: int = 20,
+        frequency_penalty: Optional[float] = None,
+        logit_bias: Optional[List[float]] = None,
+        logprobs: Optional[bool] = None,
+        max_tokens: Optional[int] = None,
+        n: Optional[int] = None,
+        presence_penalty: Optional[float] = None,
         seed: Optional[int] = None,
-        stop: Optional[Union[List[str], str]] = None,
-        temperature: float = 1.0,
+        stop: Optional[List[str]] = None,
+        temperature: Optional[float] = None,
+        tool_choice: Optional[Union[ChatCompletionInputToolTypeClass, ChatCompletionInputToolTypeEnum]] = None,
+        tool_prompt: Optional[str] = None,
+        tools: Optional[List[ChatCompletionInputTool]] = None,
+        top_logprobs: Optional[int] = None,
         top_p: Optional[float] = None,
     ) -> AsyncIterable[ChatCompletionStreamOutput]: ...
 
@@ -443,10 +469,19 @@ class AsyncInferenceClient:
         *,
         model: Optional[str] = None,
         stream: bool = False,
-        max_tokens: int = 20,
+        frequency_penalty: Optional[float] = None,
+        logit_bias: Optional[List[float]] = None,
+        logprobs: Optional[bool] = None,
+        max_tokens: Optional[int] = None,
+        n: Optional[int] = None,
+        presence_penalty: Optional[float] = None,
         seed: Optional[int] = None,
-        stop: Optional[Union[List[str], str]] = None,
-        temperature: float = 1.0,
+        stop: Optional[List[str]] = None,
+        temperature: Optional[float] = None,
+        tool_choice: Optional[Union[ChatCompletionInputToolTypeClass, ChatCompletionInputToolTypeEnum]] = None,
+        tool_prompt: Optional[str] = None,
+        tools: Optional[List[ChatCompletionInputTool]] = None,
+        top_logprobs: Optional[int] = None,
         top_p: Optional[float] = None,
     ) -> Union[ChatCompletionOutput, AsyncIterable[ChatCompletionStreamOutput]]: ...
 
@@ -456,10 +491,20 @@ class AsyncInferenceClient:
         *,
         model: Optional[str] = None,
         stream: bool = False,
-        max_tokens: int = 20,
+        # Parameters from ChatCompletionInput (handled manually)
+        frequency_penalty: Optional[float] = None,
+        logit_bias: Optional[List[float]] = None,
+        logprobs: Optional[bool] = None,
+        max_tokens: Optional[int] = None,
+        n: Optional[int] = None,
+        presence_penalty: Optional[float] = None,
         seed: Optional[int] = None,
-        stop: Optional[Union[List[str], str]] = None,
-        temperature: float = 1.0,
+        stop: Optional[List[str]] = None,
+        temperature: Optional[float] = None,
+        tool_choice: Optional[Union[ChatCompletionInputToolTypeClass, ChatCompletionInputToolTypeEnum]] = None,
+        tool_prompt: Optional[str] = None,
+        tools: Optional[List[ChatCompletionInputTool]] = None,
+        top_logprobs: Optional[int] = None,
         top_p: Optional[float] = None,
     ) -> Union[ChatCompletionOutput, AsyncIterable[ChatCompletionStreamOutput]]:
         """
@@ -482,27 +527,52 @@ class AsyncInferenceClient:
                 The model to use for chat-completion. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed
                 Inference Endpoint. If not provided, the default recommended model for chat-based text-generation will be used.
                 See https://huggingface.co/tasks/text-generation for more details.
-            frequency_penalty (`float`, optional):
+            frequency_penalty (`float`, *optional*):
                 Penalizes new tokens based on their existing frequency
                 in the text so far. Range: [-2.0, 2.0]. Defaults to 0.0.
-            max_tokens (`int`, optional):
+            logit_bias (`List[float]`, *optional*):
+                Modify the likelihood of specified tokens appearing in the completion. Accepts a JSON object that maps tokens
+                (specified by their token ID in the tokenizer) to an associated bias value from -100 to 100. Mathematically,
+                the bias is added to the logits generated by the model prior to sampling. The exact effect will vary per model,
+                but values between -1 and 1 should decrease or increase likelihood of selection; values like -100 or 100 should
+                result in a ban or exclusive selection of the relevant token. Defaults to None.
+            logprobs (`bool`, *optional*):
+                Whether to return log probabilities of the output tokens or not. If true, returns the log
+                probabilities of each output token returned in the content of message.
+            max_tokens (`int`, *optional*):
                 Maximum number of tokens allowed in the response. Defaults to 20.
-            seed (Optional[`int`], optional):
+            n (`int`, *optional*):
+                UNUSED.
+            presence_penalty (`float`, *optional*):
+                Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the
+                text so far, increasing the model's likelihood to talk about new topics.
+            seed (Optional[`int`], *optional*):
                 Seed for reproducible control flow. Defaults to None.
-            stop (Optional[`str`], optional):
+            stop (Optional[`str`], *optional*):
                 Up to four strings which trigger the end of the response.
                 Defaults to None.
-            stream (`bool`, optional):
+            stream (`bool`, *optional*):
                 Enable realtime streaming of responses. Defaults to False.
-            temperature (`float`, optional):
+            temperature (`float`, *optional*):
                 Controls randomness of the generations. Lower values ensure
                 less random completions. Range: [0, 2]. Defaults to 1.0.
-            top_p (`float`, optional):
+            top_logprobs (`int`, *optional*):
+                An integer between 0 and 5 specifying the number of most likely tokens to return at each token
+                position, each with an associated log probability. logprobs must be set to true if this parameter is
+                used.
+            top_p (`float`, *optional*):
                 Fraction of the most likely next words to sample from.
                 Must be between 0 and 1. Defaults to 1.0.
+            tool_choice ([`ChatCompletionInputToolTypeClass`] or [`ChatCompletionInputToolTypeEnum`], *optional*):
+                The tool to use for the completion. Defaults to "auto".
+            tool_prompt (`str`, *optional*):
+                A prompt to be appended before the tools.
+            tools (List of [`ChatCompletionInputTool`], *optional*):
+                A list of tools the model may call. Currently, only functions are supported as a tool. Use this to
+                provide a list of functions the model may generate JSON inputs for.
 
         Returns:
-            `Union[ChatCompletionOutput, Iterable[ChatCompletionStreamOutput]]`:
+            [`ChatCompletionOutput] or Iterable of [`ChatCompletionStreamOutput`]:
             Generated text returned from the server:
             - if `stream=False`, the generated text is returned as a [`ChatCompletionOutput`] (default).
             - if `stream=True`, the generated text is returned token by token as a sequence of [`ChatCompletionStreamOutput`].
@@ -514,19 +584,21 @@ class AsyncInferenceClient:
                 If the request fails with an HTTP error status code other than HTTP 503.
 
         Example:
+
         ```py
         # Must be run in an async context
+        # Chat example
         >>> from huggingface_hub import AsyncInferenceClient
         >>> messages = [{"role": "user", "content": "What is the capital of France?"}]
         >>> client = AsyncInferenceClient("HuggingFaceH4/zephyr-7b-beta")
         >>> await client.chat_completion(messages, max_tokens=100)
         ChatCompletionOutput(
             choices=[
-                ChatCompletionOutputChoice(
+                ChatCompletionOutputComplete(
                     finish_reason='eos_token',
                     index=0,
-                    message=ChatCompletionOutputChoiceMessage(
-                        content='The capital of France is Paris. The official name of the city is "Ville de Paris" (City of Paris) and the name of the country\'s governing body, which is located in Paris, is "La République française" (The French Republic). \nI hope that helps! Let me know if you need any further information.'
+                    message=ChatCompletionOutputMessage(
+                        content='The capital of France is Paris. The official name of the city is Ville de Paris (City of Paris) and the name of the country governing body, which is located in Paris, is La République française (The French Republic). \nI hope that helps! Let me know if you need any further information.'
                     )
                 )
             ],
@@ -539,7 +611,87 @@ class AsyncInferenceClient:
         ChatCompletionStreamOutput(choices=[ChatCompletionStreamOutputChoice(delta=ChatCompletionStreamOutputDelta(content=' capital', role='assistant'), index=0, finish_reason=None)], created=1710498504)
         (...)
         ChatCompletionStreamOutput(choices=[ChatCompletionStreamOutputChoice(delta=ChatCompletionStreamOutputDelta(content=' may', role='assistant'), index=0, finish_reason=None)], created=1710498504)
-        ChatCompletionStreamOutput(choices=[ChatCompletionStreamOutputChoice(delta=ChatCompletionStreamOutputDelta(content=None, role=None), index=0, finish_reason='length')], created=1710498504)
+
+        # Chat example with tools
+        >>> client = AsyncInferenceClient("meta-llama/Meta-Llama-3-70B-Instruct")
+        >>> messages = [
+        ...     {
+        ...         "role": "system",
+        ...         "content": "Don't make assumptions about what values to plug into functions. Ask async for clarification if a user request is ambiguous.",
+        ...     },
+        ...     {
+        ...         "role": "user",
+        ...         "content": "What's the weather like the next 3 days in San Francisco, CA?",
+        ...     },
+        ... ]
+        >>> tools = [
+        ...     {
+        ...         "type": "function",
+        ...         "function": {
+        ...             "name": "get_current_weather",
+        ...             "description": "Get the current weather",
+        ...             "parameters": {
+        ...                 "type": "object",
+        ...                 "properties": {
+        ...                     "location": {
+        ...                         "type": "string",
+        ...                         "description": "The city and state, e.g. San Francisco, CA",
+        ...                     },
+        ...                     "format": {
+        ...                         "type": "string",
+        ...                         "enum": ["celsius", "fahrenheit"],
+        ...                         "description": "The temperature unit to use. Infer this from the users location.",
+        ...                     },
+        ...                 },
+        ...                 "required": ["location", "format"],
+        ...             },
+        ...         },
+        ...     },
+        ...     {
+        ...         "type": "function",
+        ...         "function": {
+        ...             "name": "get_n_day_weather_forecast",
+        ...             "description": "Get an N-day weather forecast",
+        ...             "parameters": {
+        ...                 "type": "object",
+        ...                 "properties": {
+        ...                     "location": {
+        ...                         "type": "string",
+        ...                         "description": "The city and state, e.g. San Francisco, CA",
+        ...                     },
+        ...                     "format": {
+        ...                         "type": "string",
+        ...                         "enum": ["celsius", "fahrenheit"],
+        ...                         "description": "The temperature unit to use. Infer this from the users location.",
+        ...                     },
+        ...                     "num_days": {
+        ...                         "type": "integer",
+        ...                         "description": "The number of days to forecast",
+        ...                     },
+        ...                 },
+        ...                 "required": ["location", "format", "num_days"],
+        ...             },
+        ...         },
+        ...     },
+        ... ]
+
+        >>> response = await client.chat_completion(
+        ...     model="meta-llama/Meta-Llama-3-70B-Instruct",
+        ...     messages=messages,
+        ...     tools=tools,
+        ...     tool_choice="auto",
+        ...     max_tokens=500,
+        ... )
+        >>> response.choices[0].message.tool_calls[0].function
+        ChatCompletionOutputFunctionDefinition(
+            arguments={
+                'location': 'San Francisco, CA',
+                'format': 'fahrenheit',
+                'num_days': 3
+            },
+            name='get_n_day_weather_forecast',
+            description=None
+        )
         ```
         """
         # determine model
@@ -558,10 +710,19 @@ class AsyncInferenceClient:
                     json=dict(
                         model="tgi",  # random string
                         messages=messages,
+                        frequency_penalty=frequency_penalty,
+                        logit_bias=logit_bias,
+                        logprobs=logprobs,
                         max_tokens=max_tokens,
+                        n=n,
+                        presence_penalty=presence_penalty,
                         seed=seed,
                         stop=stop,
                         temperature=temperature,
+                        tool_choice=tool_choice,
+                        tool_prompt=tool_prompt,
+                        tools=tools,
+                        top_logprobs=top_logprobs,
                         top_p=top_p,
                         stream=stream,
                     ),
@@ -622,7 +783,6 @@ class AsyncInferenceClient:
         prompt = render_chat_prompt(model_id=model_id, token=self.token, messages=messages)
 
         # generate response
-        stop_sequences = [stop] if isinstance(stop, str) else stop
         text_generation_output = await self.text_generation(
             prompt=prompt,
             details=True,
@@ -630,7 +790,7 @@ class AsyncInferenceClient:
             model=model,
             max_new_tokens=max_tokens,
             seed=seed,
-            stop_sequences=stop_sequences,
+            stop_sequences=stop,
             temperature=temperature,
             top_p=top_p,
         )
@@ -651,12 +811,17 @@ class AsyncInferenceClient:
             finish_reason = "unk"
 
         return ChatCompletionOutput(
+            id="dummy",
+            model="dummy",
+            object="dummy",
+            system_fingerprint="dummy",
+            usage=None,  # type: ignore # set to None as we don't want to provide fake information
             created=created,
             choices=[
-                ChatCompletionOutputChoice(
+                ChatCompletionOutputComplete(
                     finish_reason=finish_reason,  # type: ignore
                     index=0,
-                    message=ChatCompletionOutputChoiceMessage(
+                    message=ChatCompletionOutputMessage(
                         content=content,
                         role="assistant",
                     ),
@@ -1507,19 +1672,24 @@ class AsyncInferenceClient:
         details: Literal[False] = ...,
         stream: Literal[False] = ...,
         model: Optional[str] = None,
-        do_sample: bool = False,
-        max_new_tokens: int = 20,
+        # Parameters from `TextGenerationInputGenerateParameters` (maintained manually)
         best_of: Optional[int] = None,
+        decoder_input_details: Optional[bool] = None,
+        do_sample: Optional[bool] = False,  # Manual default value
+        frequency_penalty: Optional[float] = None,
+        grammar: Optional[TextGenerationInputGrammarType] = None,
+        max_new_tokens: Optional[int] = None,
         repetition_penalty: Optional[float] = None,
-        return_full_text: bool = False,
+        return_full_text: Optional[bool] = False,  # Manual default value
         seed: Optional[int] = None,
-        stop_sequences: Optional[List[str]] = None,
+        stop_sequences: Optional[List[str]] = None,  # Same as `stop`
         temperature: Optional[float] = None,
         top_k: Optional[int] = None,
+        top_n_tokens: Optional[int] = None,
         top_p: Optional[float] = None,
         truncate: Optional[int] = None,
         typical_p: Optional[float] = None,
-        watermark: bool = False,
+        watermark: Optional[bool] = None,
     ) -> str: ...
 
     @overload
@@ -1530,19 +1700,24 @@ class AsyncInferenceClient:
         details: Literal[True] = ...,
         stream: Literal[False] = ...,
         model: Optional[str] = None,
-        do_sample: bool = False,
-        max_new_tokens: int = 20,
+        # Parameters from `TextGenerationInputGenerateParameters` (maintained manually)
         best_of: Optional[int] = None,
+        decoder_input_details: Optional[bool] = None,
+        do_sample: Optional[bool] = False,  # Manual default value
+        frequency_penalty: Optional[float] = None,
+        grammar: Optional[TextGenerationInputGrammarType] = None,
+        max_new_tokens: Optional[int] = None,
         repetition_penalty: Optional[float] = None,
-        return_full_text: bool = False,
+        return_full_text: Optional[bool] = False,  # Manual default value
         seed: Optional[int] = None,
-        stop_sequences: Optional[List[str]] = None,
+        stop_sequences: Optional[List[str]] = None,  # Same as `stop`
         temperature: Optional[float] = None,
         top_k: Optional[int] = None,
+        top_n_tokens: Optional[int] = None,
         top_p: Optional[float] = None,
         truncate: Optional[int] = None,
         typical_p: Optional[float] = None,
-        watermark: bool = False,
+        watermark: Optional[bool] = None,
     ) -> TextGenerationOutput: ...
 
     @overload
@@ -1553,19 +1728,24 @@ class AsyncInferenceClient:
         details: Literal[False] = ...,
         stream: Literal[True] = ...,
         model: Optional[str] = None,
-        do_sample: bool = False,
-        max_new_tokens: int = 20,
+        # Parameters from `TextGenerationInputGenerateParameters` (maintained manually)
         best_of: Optional[int] = None,
+        decoder_input_details: Optional[bool] = None,
+        do_sample: Optional[bool] = False,  # Manual default value
+        frequency_penalty: Optional[float] = None,
+        grammar: Optional[TextGenerationInputGrammarType] = None,
+        max_new_tokens: Optional[int] = None,
         repetition_penalty: Optional[float] = None,
-        return_full_text: bool = False,
+        return_full_text: Optional[bool] = False,  # Manual default value
         seed: Optional[int] = None,
-        stop_sequences: Optional[List[str]] = None,
+        stop_sequences: Optional[List[str]] = None,  # Same as `stop`
         temperature: Optional[float] = None,
         top_k: Optional[int] = None,
+        top_n_tokens: Optional[int] = None,
         top_p: Optional[float] = None,
         truncate: Optional[int] = None,
         typical_p: Optional[float] = None,
-        watermark: bool = False,
+        watermark: Optional[bool] = None,
     ) -> AsyncIterable[str]: ...
 
     @overload
@@ -1576,19 +1756,24 @@ class AsyncInferenceClient:
         details: Literal[True] = ...,
         stream: Literal[True] = ...,
         model: Optional[str] = None,
-        do_sample: bool = False,
-        max_new_tokens: int = 20,
+        # Parameters from `TextGenerationInputGenerateParameters` (maintained manually)
         best_of: Optional[int] = None,
+        decoder_input_details: Optional[bool] = None,
+        do_sample: Optional[bool] = False,  # Manual default value
+        frequency_penalty: Optional[float] = None,
+        grammar: Optional[TextGenerationInputGrammarType] = None,
+        max_new_tokens: Optional[int] = None,
         repetition_penalty: Optional[float] = None,
-        return_full_text: bool = False,
+        return_full_text: Optional[bool] = False,  # Manual default value
         seed: Optional[int] = None,
-        stop_sequences: Optional[List[str]] = None,
+        stop_sequences: Optional[List[str]] = None,  # Same as `stop`
         temperature: Optional[float] = None,
         top_k: Optional[int] = None,
+        top_n_tokens: Optional[int] = None,
         top_p: Optional[float] = None,
         truncate: Optional[int] = None,
         typical_p: Optional[float] = None,
-        watermark: bool = False,
+        watermark: Optional[bool] = None,
     ) -> AsyncIterable[TextGenerationStreamOutput]: ...
 
     @overload
@@ -1599,19 +1784,24 @@ class AsyncInferenceClient:
         details: Literal[True] = ...,
         stream: bool = ...,
         model: Optional[str] = None,
-        do_sample: bool = False,
-        max_new_tokens: int = 20,
+        # Parameters from `TextGenerationInputGenerateParameters` (maintained manually)
         best_of: Optional[int] = None,
+        decoder_input_details: Optional[bool] = None,
+        do_sample: Optional[bool] = False,  # Manual default value
+        frequency_penalty: Optional[float] = None,
+        grammar: Optional[TextGenerationInputGrammarType] = None,
+        max_new_tokens: Optional[int] = None,
         repetition_penalty: Optional[float] = None,
-        return_full_text: bool = False,
+        return_full_text: Optional[bool] = False,  # Manual default value
         seed: Optional[int] = None,
-        stop_sequences: Optional[List[str]] = None,
+        stop_sequences: Optional[List[str]] = None,  # Same as `stop`
         temperature: Optional[float] = None,
         top_k: Optional[int] = None,
+        top_n_tokens: Optional[int] = None,
         top_p: Optional[float] = None,
         truncate: Optional[int] = None,
         typical_p: Optional[float] = None,
-        watermark: bool = False,
+        watermark: Optional[bool] = None,
     ) -> Union[TextGenerationOutput, AsyncIterable[TextGenerationStreamOutput]]: ...
 
     async def text_generation(
@@ -1621,20 +1811,24 @@ class AsyncInferenceClient:
         details: bool = False,
         stream: bool = False,
         model: Optional[str] = None,
-        do_sample: bool = False,
-        max_new_tokens: int = 20,
+        # Parameters from `TextGenerationInputGenerateParameters` (maintained manually)
         best_of: Optional[int] = None,
+        decoder_input_details: Optional[bool] = None,
+        do_sample: Optional[bool] = False,  # Manual default value
+        frequency_penalty: Optional[float] = None,
+        grammar: Optional[TextGenerationInputGrammarType] = None,
+        max_new_tokens: Optional[int] = None,
         repetition_penalty: Optional[float] = None,
-        return_full_text: bool = False,
+        return_full_text: Optional[bool] = False,  # Manual default value
         seed: Optional[int] = None,
-        stop_sequences: Optional[List[str]] = None,
+        stop_sequences: Optional[List[str]] = None,  # Same as `stop`
         temperature: Optional[float] = None,
         top_k: Optional[int] = None,
+        top_n_tokens: Optional[int] = None,
         top_p: Optional[float] = None,
         truncate: Optional[int] = None,
         typical_p: Optional[float] = None,
-        watermark: bool = False,
-        decoder_input_details: bool = False,
+        watermark: Optional[bool] = None,
     ) -> Union[str, TextGenerationOutput, AsyncIterable[str], AsyncIterable[TextGenerationStreamOutput]]:
         """
         Given a prompt, generate the following text.
@@ -1662,38 +1856,46 @@ class AsyncInferenceClient:
             model (`str`, *optional*):
                 The model to use for inference. Can be a model ID hosted on the Hugging Face Hub or a URL to a deployed
                 Inference Endpoint. This parameter overrides the model defined at the instance level. Defaults to None.
-            do_sample (`bool`):
+            best_of (`int`, *optional*):
+                Generate best_of sequences and return the one if the highest token logprobs.
+            decoder_input_details (`bool`, *optional*):
+                Return the decoder input token logprobs and ids. You must set `details=True` as well for it to be taken
+                into account. Defaults to `False`.
+            do_sample (`bool`, *optional*):
                 Activate logits sampling
-            max_new_tokens (`int`):
+            frequency_penalty (`float`, *optional*):
+                Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in
+                the text so far, decreasing the model's likelihood to repeat the same line verbatim.
+            grammar ([`TextGenerationInputGrammarType`], *optional*):
+                Grammar constraints. Can be either a JSONSchema or a regex.
+            max_new_tokens (`int`, *optional*):
                 Maximum number of generated tokens
-            best_of (`int`):
-                Generate best_of sequences and return the one if the highest token logprobs
-            repetition_penalty (`float`):
+            repetition_penalty (`float`, *optional*):
                 The parameter for repetition penalty. 1.0 means no penalty. See [this
                 paper](https://arxiv.org/pdf/1909.05858.pdf) for more details.
-            return_full_text (`bool`):
+            return_full_text (`bool`, *optional*):
                 Whether to prepend the prompt to the generated text
-            seed (`int`):
+            seed (`int`, *optional*):
                 Random sampling seed
-            stop_sequences (`List[str]`):
+            stop_sequences (`List[str]`, *optional*):
                 Stop generating tokens if a member of `stop_sequences` is generated
-            temperature (`float`):
+            temperature (`float`, *optional*):
                 The value used to module the logits distribution.
-            top_k (`int`):
+            top_n_tokens (`int`, *optional*):
+                Return information about the `top_n_tokens` most likely tokens at each generation step, instead of
+                just the sampled token.
+            top_k (`int`, *optional`):
                 The number of highest probability vocabulary tokens to keep for top-k-filtering.
-            top_p (`float`):
+            top_p (`float`, *optional`):
                 If set to < 1, only the smallest set of most probable tokens with probabilities that add up to `top_p` or
                 higher are kept for generation.
-            truncate (`int`):
-                Truncate inputs tokens to the given size
-            typical_p (`float`):
+            truncate (`int`, *optional`):
+                Truncate inputs tokens to the given size.
+            typical_p (`float`, *optional`):
                 Typical Decoding mass
                 See [Typical Decoding for Natural Language Generation](https://arxiv.org/abs/2202.00666) for more information
-            watermark (`bool`):
+            watermark (`bool`, *optional`):
                 Watermarking with [A Watermark for Large Language Models](https://arxiv.org/abs/2301.10226)
-            decoder_input_details (`bool`):
-                Return the decoder input token logprobs and ids. You must set `details=True` as well for it to be taken
-                into account. Defaults to `False`.
 
         Returns:
             `Union[str, TextGenerationOutput, Iterable[str], Iterable[TextGenerationStreamOutput]]`:
@@ -1746,10 +1948,10 @@ class AsyncInferenceClient:
                 generated_tokens=12,
                 seed=None,
                 prefill=[
-                    TextGenerationPrefillToken(id=487, text='The', logprob=None),
-                    TextGenerationPrefillToken(id=53789, text=' hugging', logprob=-13.171875),
+                    TextGenerationPrefillOutputToken(id=487, text='The', logprob=None),
+                    TextGenerationPrefillOutputToken(id=53789, text=' hugging', logprob=-13.171875),
                     (...)
-                    TextGenerationPrefillToken(id=204, text=' ', logprob=-7.0390625)
+                    TextGenerationPrefillOutputToken(id=204, text=' ', logprob=-7.0390625)
                 ],
                 tokens=[
                     TokenElement(id=1425, text='100', logprob=-1.0175781, special=False),
@@ -1783,8 +1985,35 @@ class AsyncInferenceClient:
             logprob=-0.5703125,
             special=False),
             generated_text='100% open source and built to be easy to use.',
-            details=TextGenerationStreamDetails(finish_reason='length', generated_tokens=12, seed=None)
+            details=TextGenerationStreamOutputStreamDetails(finish_reason='length', generated_tokens=12, seed=None)
         )
+
+        # Case 5: generate constrained output using grammar
+        >>> response = await client.text_generation(
+        ...     prompt="I saw a puppy a cat and a raccoon during my bike ride in the park",
+        ...     model="HuggingFaceH4/zephyr-orpo-141b-A35b-v0.1",
+        ...     max_new_tokens=100,
+        ...     repetition_penalty=1.3,
+        ...     grammar={
+        ...         "type": "json",
+        ...         "value": {
+        ...             "properties": {
+        ...                 "location": {"type": "string"},
+        ...                 "activity": {"type": "string"},
+        ...                 "animals_seen": {"type": "integer", "minimum": 1, "maximum": 5},
+        ...                 "animals": {"type": "array", "items": {"type": "string"}},
+        ...             },
+        ...             "required": ["location", "activity", "animals_seen", "animals"],
+        ...         },
+        ...     },
+        ... )
+        >>> json.loads(response)
+        {
+            "activity": "bike riding",
+            "animals": ["puppy", "cat", "raccoon"],
+            "animals_seen": 3,
+            "location": "park"
+        }
         ```
         """
         if decoder_input_details and not details:
@@ -1795,41 +2024,48 @@ class AsyncInferenceClient:
             decoder_input_details = False
 
         # Build payload
+        parameters = {
+            "best_of": best_of,
+            "decoder_input_details": decoder_input_details,
+            "do_sample": do_sample,
+            "frequency_penalty": frequency_penalty,
+            "grammar": grammar,
+            "max_new_tokens": max_new_tokens,
+            "repetition_penalty": repetition_penalty,
+            "return_full_text": return_full_text,
+            "seed": seed,
+            "stop": stop_sequences if stop_sequences is not None else [],
+            "temperature": temperature,
+            "top_k": top_k,
+            "top_n_tokens": top_n_tokens,
+            "top_p": top_p,
+            "truncate": truncate,
+            "typical_p": typical_p,
+            "watermark": watermark,
+        }
+        parameters = {k: v for k, v in parameters.items() if v is not None}
         payload = {
             "inputs": prompt,
-            "parameters": {
-                "best_of": best_of,
-                "decoder_input_details": decoder_input_details,
-                "details": details,
-                "do_sample": do_sample,
-                "max_new_tokens": max_new_tokens,
-                "repetition_penalty": repetition_penalty,
-                "return_full_text": return_full_text,
-                "seed": seed,
-                "stop": stop_sequences if stop_sequences is not None else [],
-                "temperature": temperature,
-                "top_k": top_k,
-                "top_p": top_p,
-                "truncate": truncate,
-                "typical_p": typical_p,
-                "watermark": watermark,
-            },
+            "parameters": parameters,
             "stream": stream,
         }
 
         # Remove some parameters if not a TGI server
-        if not _is_tgi_server(model):
-            parameters: Dict = payload["parameters"]  # type: ignore [assignment]
+        unsupported_kwargs = _get_unsupported_text_generation_kwargs(model)
+        if len(unsupported_kwargs) > 0:
+            # The server does not support some parameters
+            # => means it is not a TGI server
+            # => remove unsupported parameters and warn the user
 
             ignored_parameters = []
-            for key in "watermark", "details", "decoder_input_details", "best_of", "stop", "return_full_text":
-                if parameters[key] is not None:
+            for key in unsupported_kwargs:
+                if parameters.get(key):
                     ignored_parameters.append(key)
-                del parameters[key]
+                parameters.pop(key, None)
             if len(ignored_parameters) > 0:
                 warnings.warn(
-                    "API endpoint/model for text-generation is not served via TGI. Ignoring parameters"
-                    f" {ignored_parameters}.",
+                    "API endpoint/model for text-generation is not served via TGI. Ignoring following parameters:"
+                    f" {', '.join(ignored_parameters)}.",
                     UserWarning,
                 )
             if details:
@@ -1849,28 +2085,32 @@ class AsyncInferenceClient:
         try:
             bytes_output = await self.post(json=payload, model=model, task="text-generation", stream=stream)  # type: ignore
         except _import_aiohttp().ClientResponseError as e:
-            error_message = getattr(e, "response_error_payload", {}).get("error", "")
-            if e.code == 400 and "The following `model_kwargs` are not used by the model" in error_message:
-                _set_as_non_tgi(model)
+            match = MODEL_KWARGS_NOT_USED_REGEX.search(e.response_error_payload["error"])
+            if e.status == 400 and match:
+                unused_params = [kwarg.strip("' ") for kwarg in match.group(1).split(",")]
+                _set_unsupported_text_generation_kwargs(model, unused_params)
                 return await self.text_generation(  # type: ignore
                     prompt=prompt,
                     details=details,
                     stream=stream,
                     model=model,
+                    best_of=best_of,
+                    decoder_input_details=decoder_input_details,
                     do_sample=do_sample,
+                    frequency_penalty=frequency_penalty,
+                    grammar=grammar,
                     max_new_tokens=max_new_tokens,
-                    best_of=best_of,
                     repetition_penalty=repetition_penalty,
                     return_full_text=return_full_text,
                     seed=seed,
                     stop_sequences=stop_sequences,
                     temperature=temperature,
                     top_k=top_k,
+                    top_n_tokens=top_n_tokens,
                     top_p=top_p,
                     truncate=truncate,
                     typical_p=typical_p,
                     watermark=watermark,
-                    decoder_input_details=decoder_input_details,
                 )
             raise_text_generation_error(e)
 
diff --git a/src/huggingface_hub/inference/_generated/types/__init__.py b/src/huggingface_hub/inference/_generated/types/__init__.py
index ece102c4..e7993037 100644
--- a/src/huggingface_hub/inference/_generated/types/__init__.py
+++ b/src/huggingface_hub/inference/_generated/types/__init__.py
@@ -19,13 +19,28 @@ from .automatic_speech_recognition import (
 from .base import BaseInferenceType
 from .chat_completion import (
     ChatCompletionInput,
+    ChatCompletionInputFunctionDefinition,
     ChatCompletionInputMessage,
+    ChatCompletionInputTool,
+    ChatCompletionInputToolCall,
+    ChatCompletionInputToolTypeClass,
     ChatCompletionOutput,
-    ChatCompletionOutputChoice,
-    ChatCompletionOutputChoiceMessage,
+    ChatCompletionOutputComplete,
+    ChatCompletionOutputFunctionDefinition,
+    ChatCompletionOutputLogprob,
+    ChatCompletionOutputLogprobs,
+    ChatCompletionOutputMessage,
+    ChatCompletionOutputToolCall,
+    ChatCompletionOutputTopLogprob,
+    ChatCompletionOutputUsage,
     ChatCompletionStreamOutput,
     ChatCompletionStreamOutputChoice,
     ChatCompletionStreamOutputDelta,
+    ChatCompletionStreamOutputDeltaToolCall,
+    ChatCompletionStreamOutputFunction,
+    ChatCompletionStreamOutputLogprob,
+    ChatCompletionStreamOutputLogprobs,
+    ChatCompletionStreamOutputTopLogprob,
 )
 from .depth_estimation import DepthEstimationInput, DepthEstimationOutput
 from .document_question_answering import (
@@ -67,14 +82,16 @@ from .text2text_generation import Text2TextGenerationInput, Text2TextGenerationO
 from .text_classification import TextClassificationInput, TextClassificationOutputElement, TextClassificationParameters
 from .text_generation import (
     TextGenerationInput,
+    TextGenerationInputGenerateParameters,
+    TextGenerationInputGrammarType,
     TextGenerationOutput,
+    TextGenerationOutputBestOfSequence,
     TextGenerationOutputDetails,
-    TextGenerationOutputSequenceDetails,
+    TextGenerationOutputPrefillToken,
     TextGenerationOutputToken,
-    TextGenerationParameters,
-    TextGenerationPrefillToken,
-    TextGenerationStreamDetails,
     TextGenerationStreamOutput,
+    TextGenerationStreamOutputStreamDetails,
+    TextGenerationStreamOutputToken,
 )
 from .text_to_audio import TextToAudioGenerationParameters, TextToAudioInput, TextToAudioOutput, TextToAudioParameters
 from .text_to_image import TextToImageInput, TextToImageOutput, TextToImageParameters, TextToImageTargetSize
diff --git a/src/huggingface_hub/inference/_generated/types/chat_completion.py b/src/huggingface_hub/inference/_generated/types/chat_completion.py
index 43e24f81..e20dc11d 100644
--- a/src/huggingface_hub/inference/_generated/types/chat_completion.py
+++ b/src/huggingface_hub/inference/_generated/types/chat_completion.py
@@ -4,103 +4,257 @@
 #   - script: https://github.com/huggingface/huggingface.js/blob/main/packages/tasks/scripts/inference-codegen.ts
 #   - specs:  https://github.com/huggingface/huggingface.js/tree/main/packages/tasks/src/tasks.
 from dataclasses import dataclass
-from typing import List, Literal, Optional, Union
+from typing import Any, List, Literal, Optional, Union
 
 from .base import BaseInferenceType
 
 
-ChatCompletionMessageRole = Literal["assistant", "system", "user"]
+@dataclass
+class ChatCompletionInputFunctionDefinition(BaseInferenceType):
+    arguments: Any
+    name: str
+    description: Optional[str] = None
+
+
+@dataclass
+class ChatCompletionInputToolCall(BaseInferenceType):
+    function: ChatCompletionInputFunctionDefinition
+    id: int
+    type: str
 
 
 @dataclass
 class ChatCompletionInputMessage(BaseInferenceType):
-    content: str
-    """The content of the message."""
-    role: "ChatCompletionMessageRole"
+    role: str
+    content: Optional[str] = None
+    name: Optional[str] = None
+    tool_calls: Optional[List[ChatCompletionInputToolCall]] = None
+
+
+@dataclass
+class ChatCompletionInputToolTypeClass(BaseInferenceType):
+    function_name: str
+
+
+ChatCompletionInputToolTypeEnum = Literal["OneOf"]
+
+
+@dataclass
+class ChatCompletionInputTool(BaseInferenceType):
+    function: ChatCompletionInputFunctionDefinition
+    type: str
 
 
 @dataclass
 class ChatCompletionInput(BaseInferenceType):
-    """Inputs for ChatCompletion inference"""
+    """Chat Completion Input.
+    Auto-generated from TGI specs.
+    For more details, check out
+    https://github.com/huggingface/huggingface.js/blob/main/packages/tasks/scripts/inference-tgi-import.ts.
+    """
 
     messages: List[ChatCompletionInputMessage]
+    """A list of messages comprising the conversation so far."""
+    model: str
+    """[UNUSED] ID of the model to use. See the model endpoint compatibility table for details
+    on which models work with the Chat API.
+    """
     frequency_penalty: Optional[float] = None
     """Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing
-    frequency in the text so far, decreasing the model's likelihood to repeat the same line
-    verbatim.
+    frequency in the text so far,
+    decreasing the model's likelihood to repeat the same line verbatim.
+    """
+    logit_bias: Optional[List[float]] = None
+    """UNUSED
+    Modify the likelihood of specified tokens appearing in the completion. Accepts a JSON
+    object that maps tokens
+    (specified by their token ID in the tokenizer) to an associated bias value from -100 to
+    100. Mathematically,
+    the bias is added to the logits generated by the model prior to sampling. The exact
+    effect will vary per model,
+    but values between -1 and 1 should decrease or increase likelihood of selection; values
+    like -100 or 100 should
+    result in a ban or exclusive selection of the relevant token.
+    """
+    logprobs: Optional[bool] = None
+    """Whether to return log probabilities of the output tokens or not. If true, returns the log
+    probabilities of each
+    output token returned in the content of message.
     """
     max_tokens: Optional[int] = None
     """The maximum number of tokens that can be generated in the chat completion."""
+    n: Optional[int] = None
+    """UNUSED
+    How many chat completion choices to generate for each input message. Note that you will
+    be charged based on the
+    number of generated tokens across all of the choices. Keep n as 1 to minimize costs.
+    """
+    presence_penalty: Optional[float] = None
+    """Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they
+    appear in the text so far,
+    increasing the model's likelihood to talk about new topics
+    """
     seed: Optional[int] = None
-    """The random sampling seed."""
-    stop: Optional[Union[List[str], str]] = None
-    """Stop generating tokens if a stop token is generated."""
+    stop: Optional[List[str]] = None
+    """Up to 4 sequences where the API will stop generating further tokens."""
     stream: Optional[bool] = None
-    """If set, partial message deltas will be sent."""
     temperature: Optional[float] = None
-    """The value used to modulate the logits distribution."""
+    """What sampling temperature to use, between 0 and 2. Higher values like 0.8 will make the
+    output more random, while
+    lower values like 0.2 will make it more focused and deterministic.
+    We generally recommend altering this or `top_p` but not both.
+    """
+    tool_choice: Optional[Union[ChatCompletionInputToolTypeClass, "ChatCompletionInputToolTypeEnum"]] = None
+    tool_prompt: Optional[str] = None
+    """A prompt to be appended before the tools"""
+    tools: Optional[List[ChatCompletionInputTool]] = None
+    """A list of tools the model may call. Currently, only functions are supported as a tool.
+    Use this to provide a list of
+    functions the model may generate JSON inputs for.
+    """
+    top_logprobs: Optional[int] = None
+    """An integer between 0 and 5 specifying the number of most likely tokens to return at each
+    token position, each with
+    an associated log probability. logprobs must be set to true if this parameter is used.
+    """
     top_p: Optional[float] = None
-    """If set to < 1, only the smallest set of most probable tokens with probabilities that add
-    up to `top_p` or higher are kept for generation.
+    """An alternative to sampling with temperature, called nucleus sampling, where the model
+    considers the results of the
+    tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10%
+    probability mass are considered.
     """
 
 
-ChatCompletionFinishReason = Literal["length", "eos_token", "stop_sequence"]
+@dataclass
+class ChatCompletionOutputTopLogprob(BaseInferenceType):
+    logprob: float
+    token: str
 
 
 @dataclass
-class ChatCompletionOutputChoiceMessage(BaseInferenceType):
-    content: str
-    """The content of the chat completion message."""
-    role: "ChatCompletionMessageRole"
+class ChatCompletionOutputLogprob(BaseInferenceType):
+    logprob: float
+    token: str
+    top_logprobs: List[ChatCompletionOutputTopLogprob]
 
 
 @dataclass
-class ChatCompletionOutputChoice(BaseInferenceType):
-    finish_reason: "ChatCompletionFinishReason"
-    """The reason why the generation was stopped."""
+class ChatCompletionOutputLogprobs(BaseInferenceType):
+    content: List[ChatCompletionOutputLogprob]
+
+
+@dataclass
+class ChatCompletionOutputFunctionDefinition(BaseInferenceType):
+    arguments: Any
+    name: str
+    description: Optional[str] = None
+
+
+@dataclass
+class ChatCompletionOutputToolCall(BaseInferenceType):
+    function: ChatCompletionOutputFunctionDefinition
+    id: int
+    type: str
+
+
+@dataclass
+class ChatCompletionOutputMessage(BaseInferenceType):
+    role: str
+    content: Optional[str] = None
+    name: Optional[str] = None
+    tool_calls: Optional[List[ChatCompletionOutputToolCall]] = None
+
+
+@dataclass
+class ChatCompletionOutputComplete(BaseInferenceType):
+    finish_reason: str
     index: int
-    """The index of the choice in the list of choices."""
-    message: ChatCompletionOutputChoiceMessage
+    message: ChatCompletionOutputMessage
+    logprobs: Optional[ChatCompletionOutputLogprobs] = None
+
+
+@dataclass
+class ChatCompletionOutputUsage(BaseInferenceType):
+    completion_tokens: int
+    prompt_tokens: int
+    total_tokens: int
 
 
 @dataclass
 class ChatCompletionOutput(BaseInferenceType):
-    """Outputs for Chat Completion inference"""
+    """Chat Completion Output.
+    Auto-generated from TGI specs.
+    For more details, check out
+    https://github.com/huggingface/huggingface.js/blob/main/packages/tasks/scripts/inference-tgi-import.ts.
+    """
 
-    choices: List[ChatCompletionOutputChoice]
-    """A list of chat completion choices."""
+    choices: List[ChatCompletionOutputComplete]
     created: int
-    """The Unix timestamp (in seconds) of when the chat completion was created."""
+    id: str
+    model: str
+    object: str
+    system_fingerprint: str
+    usage: ChatCompletionOutputUsage
 
 
 @dataclass
-class ChatCompletionStreamOutputDelta(BaseInferenceType):
-    """A chat completion delta generated by streamed model responses."""
+class ChatCompletionStreamOutputFunction(BaseInferenceType):
+    arguments: str
+    name: Optional[str] = None
+
+
+@dataclass
+class ChatCompletionStreamOutputDeltaToolCall(BaseInferenceType):
+    function: ChatCompletionStreamOutputFunction
+    id: str
+    index: int
+    type: str
 
+
+@dataclass
+class ChatCompletionStreamOutputDelta(BaseInferenceType):
+    role: str
     content: Optional[str] = None
-    """The contents of the chunk message."""
-    role: Optional[str] = None
-    """The role of the author of this message."""
+    tool_calls: Optional[ChatCompletionStreamOutputDeltaToolCall] = None
+
+
+@dataclass
+class ChatCompletionStreamOutputTopLogprob(BaseInferenceType):
+    logprob: float
+    token: str
+
+
+@dataclass
+class ChatCompletionStreamOutputLogprob(BaseInferenceType):
+    logprob: float
+    token: str
+    top_logprobs: List[ChatCompletionStreamOutputTopLogprob]
+
+
+@dataclass
+class ChatCompletionStreamOutputLogprobs(BaseInferenceType):
+    content: List[ChatCompletionStreamOutputLogprob]
 
 
 @dataclass
 class ChatCompletionStreamOutputChoice(BaseInferenceType):
     delta: ChatCompletionStreamOutputDelta
-    """A chat completion delta generated by streamed model responses."""
     index: int
-    """The index of the choice in the list of choices."""
-    finish_reason: Optional["ChatCompletionFinishReason"] = None
-    """The reason why the generation was stopped."""
+    finish_reason: Optional[str] = None
+    logprobs: Optional[ChatCompletionStreamOutputLogprobs] = None
 
 
 @dataclass
 class ChatCompletionStreamOutput(BaseInferenceType):
-    """Chat Completion Stream Output"""
+    """Chat Completion Stream Output.
+    Auto-generated from TGI specs.
+    For more details, check out
+    https://github.com/huggingface/huggingface.js/blob/main/packages/tasks/scripts/inference-tgi-import.ts.
+    """
 
     choices: List[ChatCompletionStreamOutputChoice]
-    """A list of chat completion choices."""
     created: int
-    """The Unix timestamp (in seconds) of when the chat completion was created. Each chunk has
-    the same timestamp.
-    """
+    id: str
+    model: str
+    object: str
+    system_fingerprint: str
diff --git a/src/huggingface_hub/inference/_generated/types/text_generation.py b/src/huggingface_hub/inference/_generated/types/text_generation.py
index 28669850..0d630725 100644
--- a/src/huggingface_hub/inference/_generated/types/text_generation.py
+++ b/src/huggingface_hub/inference/_generated/types/text_generation.py
@@ -4,158 +4,136 @@
 #   - script: https://github.com/huggingface/huggingface.js/blob/main/packages/tasks/scripts/inference-codegen.ts
 #   - specs:  https://github.com/huggingface/huggingface.js/tree/main/packages/tasks/src/tasks.
 from dataclasses import dataclass
-from typing import List, Literal, Optional
+from typing import Any, List, Literal, Optional
 
 from .base import BaseInferenceType
 
 
+TypeEnum = Literal["json", "regex"]
+
+
 @dataclass
-class TextGenerationParameters(BaseInferenceType):
-    """Additional inference parameters
-    Additional inference parameters for Text Generation
+class TextGenerationInputGrammarType(BaseInferenceType):
+    type: "TypeEnum"
+    value: Any
+    """A string that represents a [JSON Schema](https://json-schema.org/).
+    JSON Schema is a declarative language that allows to annotate JSON documents
+    with types and descriptions.
     """
 
+
+@dataclass
+class TextGenerationInputGenerateParameters(BaseInferenceType):
     best_of: Optional[int] = None
-    """The number of sampling queries to run. Only the best one (in terms of total logprob) will
-    be returned.
-    """
     decoder_input_details: Optional[bool] = None
-    """Whether or not to output decoder input details"""
     details: Optional[bool] = None
-    """Whether or not to output details"""
     do_sample: Optional[bool] = None
-    """Whether to use logits sampling instead of greedy decoding when generating new tokens."""
+    frequency_penalty: Optional[float] = None
+    grammar: Optional[TextGenerationInputGrammarType] = None
     max_new_tokens: Optional[int] = None
-    """The maximum number of tokens to generate."""
     repetition_penalty: Optional[float] = None
-    """The parameter for repetition penalty. A value of 1.0 means no penalty. See [this
-    paper](https://hf.co/papers/1909.05858) for more details.
-    """
     return_full_text: Optional[bool] = None
-    """Whether to prepend the prompt to the generated text."""
     seed: Optional[int] = None
-    """The random sampling seed."""
-    stop_sequences: Optional[List[str]] = None
-    """Stop generating tokens if a member of `stop_sequences` is generated."""
+    stop: Optional[List[str]] = None
     temperature: Optional[float] = None
-    """The value used to modulate the logits distribution."""
     top_k: Optional[int] = None
-    """The number of highest probability vocabulary tokens to keep for top-k-filtering."""
+    top_n_tokens: Optional[int] = None
     top_p: Optional[float] = None
-    """If set to < 1, only the smallest set of most probable tokens with probabilities that add
-    up to `top_p` or higher are kept for generation.
-    """
     truncate: Optional[int] = None
-    """Truncate input tokens to the given size."""
     typical_p: Optional[float] = None
-    """Typical Decoding mass. See [Typical Decoding for Natural Language
-    Generation](https://hf.co/papers/2202.00666) for more information
-    """
     watermark: Optional[bool] = None
-    """Watermarking with [A Watermark for Large Language Models](https://hf.co/papers/2301.10226)"""
 
 
 @dataclass
 class TextGenerationInput(BaseInferenceType):
-    """Inputs for Text Generation inference"""
+    """Text Generation Input.
+    Auto-generated from TGI specs.
+    For more details, check out
+    https://github.com/huggingface/huggingface.js/blob/main/packages/tasks/scripts/inference-tgi-import.ts.
+    """
 
     inputs: str
-    """The text to initialize generation with"""
-    parameters: Optional[TextGenerationParameters] = None
-    """Additional inference parameters"""
+    parameters: Optional[TextGenerationInputGenerateParameters] = None
     stream: Optional[bool] = None
-    """Whether to stream output tokens"""
 
 
-TextGenerationFinishReason = Literal["length", "eos_token", "stop_sequence"]
+TextGenerationOutputFinishReason = Literal["length", "eos_token", "stop_sequence"]
 
 
 @dataclass
-class TextGenerationPrefillToken(BaseInferenceType):
+class TextGenerationOutputPrefillToken(BaseInferenceType):
     id: int
     logprob: float
     text: str
-    """The text associated with that token"""
 
 
 @dataclass
 class TextGenerationOutputToken(BaseInferenceType):
-    """Generated token."""
-
     id: int
+    logprob: float
     special: bool
-    """Whether or not that token is a special one"""
     text: str
-    """The text associated with that token"""
-    logprob: Optional[float] = None
 
 
 @dataclass
-class TextGenerationOutputSequenceDetails(BaseInferenceType):
-    finish_reason: "TextGenerationFinishReason"
+class TextGenerationOutputBestOfSequence(BaseInferenceType):
+    finish_reason: "TextGenerationOutputFinishReason"
     generated_text: str
-    """The generated text"""
     generated_tokens: int
-    """The number of generated tokens"""
-    prefill: List[TextGenerationPrefillToken]
+    prefill: List[TextGenerationOutputPrefillToken]
     tokens: List[TextGenerationOutputToken]
-    """The generated tokens and associated details"""
     seed: Optional[int] = None
-    """The random seed used for generation"""
     top_tokens: Optional[List[List[TextGenerationOutputToken]]] = None
-    """Most likely tokens"""
 
 
 @dataclass
 class TextGenerationOutputDetails(BaseInferenceType):
-    """When enabled, details about the generation"""
-
-    finish_reason: "TextGenerationFinishReason"
-    """The reason why the generation was stopped."""
+    finish_reason: "TextGenerationOutputFinishReason"
     generated_tokens: int
-    """The number of generated tokens"""
-    prefill: List[TextGenerationPrefillToken]
+    prefill: List[TextGenerationOutputPrefillToken]
     tokens: List[TextGenerationOutputToken]
-    """The generated tokens and associated details"""
-    best_of_sequences: Optional[List[TextGenerationOutputSequenceDetails]] = None
-    """Details about additional sequences when best_of is provided"""
+    best_of_sequences: Optional[List[TextGenerationOutputBestOfSequence]] = None
     seed: Optional[int] = None
-    """The random seed used for generation"""
     top_tokens: Optional[List[List[TextGenerationOutputToken]]] = None
-    """Most likely tokens"""
 
 
 @dataclass
 class TextGenerationOutput(BaseInferenceType):
-    """Outputs for Text Generation inference"""
+    """Text Generation Output.
+    Auto-generated from TGI specs.
+    For more details, check out
+    https://github.com/huggingface/huggingface.js/blob/main/packages/tasks/scripts/inference-tgi-import.ts.
+    """
 
     generated_text: str
-    """The generated text"""
     details: Optional[TextGenerationOutputDetails] = None
-    """When enabled, details about the generation"""
 
 
 @dataclass
-class TextGenerationStreamDetails(BaseInferenceType):
-    """Generation details. Only available when the generation is finished."""
-
-    finish_reason: "TextGenerationFinishReason"
-    """The reason why the generation was stopped."""
+class TextGenerationStreamOutputStreamDetails(BaseInferenceType):
+    finish_reason: "TextGenerationOutputFinishReason"
     generated_tokens: int
-    """The number of generated tokens"""
-    seed: int
-    """The random seed used for generation"""
+    seed: Optional[int] = None
+
+
+@dataclass
+class TextGenerationStreamOutputToken(BaseInferenceType):
+    id: int
+    logprob: float
+    special: bool
+    text: str
 
 
 @dataclass
 class TextGenerationStreamOutput(BaseInferenceType):
-    """Text Generation Stream Output"""
+    """Text Generation Stream Output.
+    Auto-generated from TGI specs.
+    For more details, check out
+    https://github.com/huggingface/huggingface.js/blob/main/packages/tasks/scripts/inference-tgi-import.ts.
+    """
 
-    token: TextGenerationOutputToken
-    """Generated token."""
-    details: Optional[TextGenerationStreamDetails] = None
-    """Generation details. Only available when the generation is finished."""
+    index: int
+    token: TextGenerationStreamOutputToken
+    details: Optional[TextGenerationStreamOutputStreamDetails] = None
     generated_text: Optional[str] = None
-    """The complete generated text. Only available when the generation is finished."""
-    index: Optional[int] = None
-    """The token index within the stream. Optional to support older clients that omit it."""
+    top_tokens: Optional[List[TextGenerationStreamOutputToken]] = None
diff --git a/utils/check_inference_input_params.py b/utils/check_inference_input_params.py
new file mode 100644
index 00000000..af0945d2
--- /dev/null
+++ b/utils/check_inference_input_params.py
@@ -0,0 +1,115 @@
+# coding=utf-8
+# Copyright 2024-present, the HuggingFace Inc. team.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""Utility script to check consistency between input parameters of InferenceClient methods and generated types.
+
+TODO: check all methods
+TODO: check parameters types
+TODO: check parameters default values
+TODO: check parameters (type, description) are consistent in the docstrings
+TODO: (low priority) automatically generate the input types from the methods
+"""
+
+import inspect
+from dataclasses import is_dataclass
+from typing import Any, get_args
+
+from huggingface_hub import InferenceClient
+from huggingface_hub.inference._generated import types
+
+
+METHODS_TO_SKIP = [
+    # + all private methods
+    "post",
+    "conversational",
+]
+PARAMETERS_TO_SKIP = {
+    "chat_completion": {},
+    "text_generation": {
+        "stop",  # stop_sequence instead (for legacy reasons)
+    },
+}
+UNDOCUMENTED_PARAMETERS = {
+    "self",
+}
+
+
+def check_method(method_name: str, method: Any):
+    input_type_name = "".join(part.capitalize() for part in method_name.split("_")) + "Input"
+    if not hasattr(types, input_type_name):
+        return [f"Missing input type for method {method_name}"]
+
+    input_type = getattr(types, input_type_name)
+    docstring = method.__doc__
+
+    if method_name == "chat_completion":
+        # Special case for chat_completion
+        parameters_type = input_type
+    else:
+        parameters_field = input_type.__dataclass_fields__.get("parameters", None)
+        if parameters_field is None:
+            return [f"Missing 'parameters' field for type {input_type}"]
+
+        parameters_type = get_args(parameters_field.type)[0]
+        if not is_dataclass(parameters_type):
+            return [f"'parameters' field is not a dataclass for type {input_type} ({parameters_type})"]
+
+    # For each expected parameter, check it is defined
+    logs = []
+    method_params = inspect.signature(method).parameters
+    for param_name in parameters_type.__dataclass_fields__:
+        if param_name in PARAMETERS_TO_SKIP[method_name]:
+            continue
+        if param_name not in method_params:
+            logs.append(f"Missing parameter {param_name} in method signature")
+
+    # Check parameter is documented in docstring
+    for param_name in method_params:
+        if param_name in UNDOCUMENTED_PARAMETERS:
+            continue
+        if param_name in PARAMETERS_TO_SKIP[method_name]:
+            continue
+        if f"            {param_name} (" not in docstring:
+            logs.append(f"Parameter {param_name} is not documented")
+
+    return logs
+
+
+# Inspect InferenceClient methods individually
+exit_code = 0
+all_logs = []  # print details only if errors are found
+for method_name, method in inspect.getmembers(InferenceClient, predicate=inspect.isfunction):
+    if method_name.startswith("_") or method_name in METHODS_TO_SKIP:
+        continue
+    if method_name not in PARAMETERS_TO_SKIP:
+        all_logs.append(f"  ⏩️ {method_name}: skipped")
+        continue
+
+    logs = check_method(method_name, method)
+    if len(logs) > 0:
+        exit_code = 1
+        all_logs.append(f"  ❌ {method_name}: errors found")
+        all_logs.append("\n".join(" " * 4 + log for log in logs))
+        continue
+    else:
+        all_logs.append(f"  ✅ {method_name}: success!")
+        continue
+
+if exit_code == 0:
+    print("✅ All good! (inference inputs)")
+else:
+    print("❌ Inconsistency found in inference inputs.")
+    for log in all_logs:
+        print(log)
+exit(exit_code)
diff --git a/utils/generate_async_inference_client.py b/utils/generate_async_inference_client.py
index e787ddad..e3bcccc0 100644
--- a/utils/generate_async_inference_client.py
+++ b/utils/generate_async_inference_client.py
@@ -287,12 +287,13 @@ def _adapt_text_generation_to_async(code: str) -> str:
     code = code.replace(
         """
         except HTTPError as e:
-            if isinstance(e, BadRequestError) and "The following `model_kwargs` are not used by the model" in str(e):
+            match = MODEL_KWARGS_NOT_USED_REGEX.search(str(e))
+            if isinstance(e, BadRequestError) and match:
     """,
         """
         except _import_aiohttp().ClientResponseError as e:
-            error_message = getattr(e, "response_error_payload", {}).get("error", "")
-            if e.code == 400 and "The following `model_kwargs` are not used by the model" in error_message:
+            match = MODEL_KWARGS_NOT_USED_REGEX.search(e.response_error_payload["error"])
+            if e.status == 400 and match:
     """,
     )
 
diff --git a/utils/generate_inference_types.py b/utils/generate_inference_types.py
index f54162f4..7508808a 100644
--- a/utils/generate_inference_types.py
+++ b/utils/generate_inference_types.py
@@ -19,7 +19,7 @@ import os
 import re
 import tempfile
 from pathlib import Path
-from typing import Dict, List, NoReturn
+from typing import Dict, List, Literal, NoReturn
 
 from helpers import check_and_update_file_content
 from ruff.__main__ import find_ruff_bin
@@ -28,9 +28,12 @@ from ruff.__main__ import find_ruff_bin
 huggingface_hub_folder_path = Path(__file__).parents[1] / "src" / "huggingface_hub"
 INFERENCE_TYPES_FOLDER_PATH = huggingface_hub_folder_path / "inference" / "_generated" / "types"
 MAIN_INIT_PY_FILE = huggingface_hub_folder_path / "__init__.py"
-REFERENCE_PACKAGE_PATH = (
+REFERENCE_PACKAGE_EN_PATH = (
     Path(__file__).parents[1] / "docs" / "source" / "en" / "package_reference" / "inference_types.md"
 )
+REFERENCE_PACKAGE_KO_PATH = (
+    Path(__file__).parents[1] / "docs" / "source" / "ko" / "package_reference" / "inference_types.md"
+)
 
 IGNORE_FILES = [
     "__init__.py",
@@ -85,7 +88,7 @@ SHARED_CLASSES = [
     "TargetSize",
 ]
 
-REFERENCE_PACKAGE_CONTENT = """
+REFERENCE_PACKAGE_EN_CONTENT = """
 <!--⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be
 rendered properly in your Markdown viewer.
 -->
@@ -103,6 +106,26 @@ to find the JSON schemas for each task.
 This part of the lib is still under development and will be improved in future releases.
 
 
+{types}
+"""
+
+REFERENCE_PACKAGE_KO_CONTENT = """
+<!--⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be
+rendered properly in your Markdown viewer.
+-->
+<!--⚠️ Note that this file is auto-generated by `utils/generate_inference_types.py`. Do not modify it manually.-->
+
+
+# 추론 타입[[inference-types]]
+
+이 페이지에는 Hugging Face Hub에서 지원하는 타입(예: 데이터 클래스)이 나열되어 있습니다.
+각 작업은 JSON 스키마를 사용하여 지정되며, 이러한 스키마에 의해서 타입이 생성됩니다. 이때 Python 요구 사항으로 인해 일부 사용자 정의가 있을 수 있습니다.
+
+각 작업의 JSON 스키마를 확인하려면 [@huggingface.js/tasks](https://github.com/huggingface/huggingface.js/tree/main/packages/tasks/src/tasks)를 확인하세요.
+
+라이브러리에서 이 부분은 아직 개발 중이며, 향후 릴리즈에서 개선될 예정입니다.
+
+
 {types}
 """
 
@@ -215,16 +238,24 @@ def format_source_code(code: str) -> str:
         return filepath.read_text()
 
 
-def generate_reference_package(dataclasses: Dict[str, List[str]]) -> str:
+def generate_reference_package(dataclasses: Dict[str, List[str]], language: Literal["en", "ko"]) -> str:
     """Generate the reference package content."""
 
     per_task_docs = []
     for task in sorted(dataclasses.keys()):
         lines = [f"[[autodoc]] huggingface_hub.{cls}" for cls in sorted(dataclasses[task])]
         lines_str = "\n\n".join(lines)
-        per_task_docs.append(f"\n## {task}\n\n{lines_str}\n\n")
+        if language == "en":
+            # e.g. '## audio_classification'
+            per_task_docs.append(f"\n## {task}\n\n{lines_str}\n\n")
+        elif language == "ko":
+            # e.g. '## audio_classification[[huggingface_hub.AudioClassificationInput]]'
+            per_task_docs.append(f"\n## {task}[[huggingface_hub.{sorted(dataclasses[task])[0]}]]\n\n{lines_str}\n\n")
+        else:
+            raise ValueError(f"Language {language} is not supported.")
 
-    return REFERENCE_PACKAGE_CONTENT.format(types="\n".join(per_task_docs))
+    template = REFERENCE_PACKAGE_EN_CONTENT if language == "en" else REFERENCE_PACKAGE_KO_CONTENT
+    return template.format(types="\n".join(per_task_docs))
 
 
 def check_inference_types(update: bool) -> NoReturn:
@@ -256,8 +287,11 @@ def check_inference_types(update: bool) -> NoReturn:
     updated_main_init_py_content = format_source_code(updated_main_init_py_content)
     check_and_update_file_content(MAIN_INIT_PY_FILE, updated_main_init_py_content, update)
 
-    reference_package_content = generate_reference_package(dataclasses)
-    check_and_update_file_content(REFERENCE_PACKAGE_PATH, reference_package_content, update)
+    reference_package_content_en = generate_reference_package(dataclasses, "en")
+    check_and_update_file_content(REFERENCE_PACKAGE_EN_PATH, reference_package_content_en, update)
+
+    reference_package_content_ko = generate_reference_package(dataclasses, "ko")
+    check_and_update_file_content(REFERENCE_PACKAGE_KO_PATH, reference_package_content_ko, update)
 
     print("✅ All good! (inference types)")
     exit(0)


Test patch:
diff --git a/tests/cassettes/InferenceClientVCRTest.test_chat_completion_with_tool.yaml b/tests/cassettes/InferenceClientVCRTest.test_chat_completion_with_tool.yaml
new file mode 100644
index 00000000..cc815993
--- /dev/null
+++ b/tests/cassettes/InferenceClientVCRTest.test_chat_completion_with_tool.yaml
@@ -0,0 +1,152 @@
+interactions:
+- request:
+    body: '{"model": "tgi", "messages": [{"role": "system", "content": "Don''t make
+      assumptions about what values to plug into functions. Ask for clarification
+      if a user request is ambiguous."}, {"role": "user", "content": "What''s the
+      weather like the next 3 days in San Francisco, CA?"}], "frequency_penalty":
+      null, "logit_bias": null, "logprobs": null, "max_tokens": 500, "n": null, "presence_penalty":
+      null, "seed": null, "stop": null, "temperature": null, "tool_choice": "auto",
+      "tool_prompt": null, "tools": [{"type": "function", "function": {"name": "get_current_weather",
+      "description": "Get the current weather", "parameters": {"type": "object", "properties":
+      {"location": {"type": "string", "description": "The city and state, e.g. San
+      Francisco, CA"}, "format": {"type": "string", "enum": ["celsius", "fahrenheit"],
+      "description": "The temperature unit to use. Infer this from the users location."}},
+      "required": ["location", "format"]}}}, {"type": "function", "function": {"name":
+      "get_n_day_weather_forecast", "description": "Get an N-day weather forecast",
+      "parameters": {"type": "object", "properties": {"location": {"type": "string",
+      "description": "The city and state, e.g. San Francisco, CA"}, "format": {"type":
+      "string", "enum": ["celsius", "fahrenheit"], "description": "The temperature
+      unit to use. Infer this from the users location."}, "num_days": {"type": "integer",
+      "description": "The number of days to forecast"}}, "required": ["location",
+      "format", "num_days"]}}}], "top_logprobs": null, "top_p": null, "stream": false}'
+    headers:
+      Accept:
+      - '*/*'
+      Accept-Encoding:
+      - gzip, deflate, br
+      Connection:
+      - keep-alive
+      Content-Length:
+      - '1535'
+      Content-Type:
+      - application/json
+      X-Amzn-Trace-Id:
+      - e10745bb-413f-4524-9c8c-b1ebfa08cfa8
+      user-agent:
+      - unknown/None; hf_hub/0.23.0.dev0; python/3.10.12; torch/2.2.1; tensorflow/2.15.0.post1;
+        fastcore/1.5.23
+    method: POST
+    uri: https://api-inference.huggingface.co/models/meta-llama/Meta-Llama-3-70B-Instruct/v1/chat/completions
+  response:
+    body:
+      string: '{"id":"","object":"text_completion","created":1713800152,"model":"hsramall/hsramall-70b-chat-placeholder","system_fingerprint":"1.4.5-native","choices":[{"index":0,"message":{"role":"assistant","tool_calls":[{"id":0,"type":"function","function":{"description":null,"name":"get_n_day_weather_forecast","arguments":{"format":"fahrenheit","location":"San
+        Francisco, CA","num_days":3}}}]},"logprobs":null,"finish_reason":"eos_token"}],"usage":{"prompt_tokens":246,"completion_tokens":27,"total_tokens":273}}'
+    headers:
+      Connection:
+      - keep-alive
+      Content-Length:
+      - '483'
+      Content-Type:
+      - application/json
+      Date:
+      - Mon, 22 Apr 2024 15:48:55 GMT
+      access-control-allow-credentials:
+      - 'true'
+      vary:
+      - Origin, Access-Control-Request-Method, Access-Control-Request-Headers
+      x-compute-time:
+      - '22.899515643'
+      x-compute-type:
+      - cache
+      x-request-id:
+      - MnKOOHNB80yo4t1PKEPsP
+      x-sha:
+      - 5fcb2901844dde3111159f24205b71c25900ffbd
+    status:
+      code: 200
+      message: OK
+- request:
+    body: '{"model": "tgi", "messages": [{"role": "system", "content": "Don''t make
+      assumptions about what values to plug into functions. Ask for clarification
+      if a user request is ambiguous."}, {"role": "user", "content": "What''s the
+      weather like the next 3 days in San Francisco, CA?"}], "frequency_penalty":
+      null, "logit_bias": null, "logprobs": null, "max_tokens": 500, "n": null, "presence_penalty":
+      null, "seed": null, "stop": null, "temperature": null, "tool_choice": "get_current_weather",
+      "tool_prompt": null, "tools": [{"type": "function", "function": {"name": "get_current_weather",
+      "description": "Get the current weather", "parameters": {"type": "object", "properties":
+      {"location": {"type": "string", "description": "The city and state, e.g. San
+      Francisco, CA"}, "format": {"type": "string", "enum": ["celsius", "fahrenheit"],
+      "description": "The temperature unit to use. Infer this from the users location."}},
+      "required": ["location", "format"]}}}, {"type": "function", "function": {"name":
+      "get_n_day_weather_forecast", "description": "Get an N-day weather forecast",
+      "parameters": {"type": "object", "properties": {"location": {"type": "string",
+      "description": "The city and state, e.g. San Francisco, CA"}, "format": {"type":
+      "string", "enum": ["celsius", "fahrenheit"], "description": "The temperature
+      unit to use. Infer this from the users location."}, "num_days": {"type": "integer",
+      "description": "The number of days to forecast"}}, "required": ["location",
+      "format", "num_days"]}}}], "top_logprobs": null, "top_p": null, "stream": false}'
+    headers:
+      Accept:
+      - '*/*'
+      Accept-Encoding:
+      - gzip, deflate, br
+      Connection:
+      - keep-alive
+      Content-Length:
+      - '1550'
+      Content-Type:
+      - application/json
+      X-Amzn-Trace-Id:
+      - 8c946c95-d5eb-4413-9c01-9395d15519a8
+      user-agent:
+      - unknown/None; hf_hub/0.23.0.dev0; python/3.10.12; torch/2.2.1; tensorflow/2.15.0.post1;
+        fastcore/1.5.23
+    method: POST
+    uri: https://api-inference.huggingface.co/models/meta-llama/Meta-Llama-3-70B-Instruct/v1/chat/completions
+  response:
+    body:
+      string: '{"id":"","object":"text_completion","created":1713800953,"model":"hsramall/hsramall-70b-chat-placeholder","system_fingerprint":"1.4.5-native","choices":[{"index":0,"message":{"role":"assistant","tool_calls":[{"id":0,"type":"function","function":{"description":null,"name":"get_current_weather","arguments":{"format":"fahrenheit","location":"San
+        Francisco, CA"}}}]},"logprobs":null,"finish_reason":"eos_token"}],"usage":{"prompt_tokens":144,"completion_tokens":16,"total_tokens":160}}'
+    headers:
+      Connection:
+      - keep-alive
+      Content-Type:
+      - application/json
+      Date:
+      - Mon, 22 Apr 2024 15:49:13 GMT
+      Transfer-Encoding:
+      - chunked
+      access-control-allow-credentials:
+      - 'true'
+      access-control-allow-origin:
+      - '*'
+      vary:
+      - origin, Origin, Access-Control-Request-Method, Access-Control-Request-Headers
+      x-compute-characters:
+      - '763'
+      x-compute-time:
+      - '18.226054244'
+      x-compute-type:
+      - 8-nvidia-h100-80gb-hbm3
+      x-generated-tokens:
+      - '16'
+      x-inference-time:
+      - '17740'
+      x-prompt-tokens:
+      - '144'
+      x-queue-time:
+      - '458'
+      x-request-id:
+      - 8lseSEbt80Xtv771O-BY9
+      x-sha:
+      - 5fcb2901844dde3111159f24205b71c25900ffbd
+      x-time-per-token:
+      - '1108'
+      x-total-time:
+      - '18226'
+      x-validation-time:
+      - '26'
+    status:
+      code: 200
+      message: OK
+version: 1
diff --git a/tests/cassettes/TestTextGenerationClientVCR.test_generate_non_tgi_endpoint.yaml b/tests/cassettes/TestTextGenerationClientVCR.test_generate_non_tgi_endpoint.yaml
index dccb99f5..33b47420 100644
--- a/tests/cassettes/TestTextGenerationClientVCR.test_generate_non_tgi_endpoint.yaml
+++ b/tests/cassettes/TestTextGenerationClientVCR.test_generate_non_tgi_endpoint.yaml
@@ -24,10 +24,10 @@ interactions:
   response:
     body:
       string: '{"error":"The following `model_kwargs` are not used by the model: [''watermark'',
-        ''stop'', ''details'', ''decoder_input_details''] (note: typos in the generate
+        ''stop'', ''decoder_input_details''] (note: typos in the generate
         arguments will also show up in this list)","warnings":["There was an inference
         error: The following `model_kwargs` are not used by the model: [''watermark'',
-        ''stop'', ''details'', ''decoder_input_details''] (note: typos in the generate
+        ''stop'', ''decoder_input_details''] (note: typos in the generate
         arguments will also show up in this list)"]}'
     headers:
       Connection:
diff --git a/tests/cassettes/TestTextGenerationClientVCR.test_generate_non_tgi_endpoint_regression_test.yaml b/tests/cassettes/TestTextGenerationClientVCR.test_generate_non_tgi_endpoint_regression_test.yaml
index cda4c9a3..4864c85c 100644
--- a/tests/cassettes/TestTextGenerationClientVCR.test_generate_non_tgi_endpoint_regression_test.yaml
+++ b/tests/cassettes/TestTextGenerationClientVCR.test_generate_non_tgi_endpoint_regression_test.yaml
@@ -26,11 +26,10 @@ interactions:
   response:
     body:
       string: '{"error":"The following `model_kwargs` are not used by the model: [''return_full_text'',
-        ''watermark'', ''stop'', ''decoder_input_details'', ''details''] (note: typos
+        ''watermark'', ''stop'', ''decoder_input_details''] (note: typos
         in the generate arguments will also show up in this list)","warnings":["There
         was an inference error: The following `model_kwargs` are not used by the model:
-        [''return_full_text'', ''watermark'', ''stop'', ''decoder_input_details'',
-        ''details''] (note: typos in the generate arguments will also show up in this
+        [''return_full_text'', ''watermark'', ''stop'', ''decoder_input_details''] (note: typos in the generate arguments will also show up in this
         list)"]}'
     headers:
       Connection:
diff --git a/tests/cassettes/TestTextGenerationClientVCR.test_generate_with_grammar.yaml b/tests/cassettes/TestTextGenerationClientVCR.test_generate_with_grammar.yaml
new file mode 100644
index 00000000..51d33d76
--- /dev/null
+++ b/tests/cassettes/TestTextGenerationClientVCR.test_generate_with_grammar.yaml
@@ -0,0 +1,74 @@
+interactions:
+- request:
+    body: '{"inputs": "I saw a puppy a cat and a raccoon during my bike ride in the
+      park", "parameters": {"do_sample": false, "grammar": {"type": "json", "value":
+      {"properties": {"location": {"type": "string"}, "activity": {"type": "string"},
+      "animals_seen": {"type": "integer", "minimum": 1, "maximum": 5}, "animals":
+      {"type": "array", "items": {"type": "string"}}}, "required": ["location", "activity",
+      "animals_seen", "animals"]}}, "max_new_tokens": 100, "repetition_penalty": 1.3,
+      "return_full_text": false, "stop": []}, "stream": false}'
+    headers:
+      Accept:
+      - '*/*'
+      Accept-Encoding:
+      - gzip, deflate, br
+      Connection:
+      - keep-alive
+      Content-Length:
+      - '530'
+      Content-Type:
+      - application/json
+      X-Amzn-Trace-Id:
+      - 2b4f4146-2bcb-4d92-9261-00bef0d1908f
+      user-agent:
+      - unknown/None; hf_hub/0.23.0.dev0; python/3.10.12; torch/2.2.1; tensorflow/2.15.0.post1;
+        fastcore/1.5.23
+    method: POST
+    uri: https://api-inference.huggingface.co/models/HuggingFaceH4/zephyr-orpo-141b-A35b-v0.1
+  response:
+    body:
+      string: '[{"generated_text":"{\n\n    \"activity\" :  \"biking\",\n     \"animals\":
+        [   ] ,\n      \"animals_seen\":3,\n       \"location\":\"park\"}"}]'
+    headers:
+      Connection:
+      - keep-alive
+      Content-Type:
+      - application/json
+      Date:
+      - Fri, 19 Apr 2024 15:12:46 GMT
+      Transfer-Encoding:
+      - chunked
+      access-control-allow-credentials:
+      - 'true'
+      access-control-allow-origin:
+      - '*'
+      vary:
+      - origin, Origin, Access-Control-Request-Method, Access-Control-Request-Headers
+      x-compute-characters:
+      - '65'
+      x-compute-time:
+      - '19.325953045'
+      x-compute-type:
+      - 8-nvidia-h100-80gb-hbm3
+      x-generated-tokens:
+      - '46'
+      x-inference-time:
+      - '19286'
+      x-prompt-tokens:
+      - '20'
+      x-queue-time:
+      - '4'
+      x-request-id:
+      - 7-SN-pShdAwPxAFjZkyIh
+      x-sha:
+      - a3be084543d278e61b64cd600f28157afc79ffd3
+      x-time-per-token:
+      - '419'
+      x-total-time:
+      - '19325'
+      x-validation-time:
+      - '34'
+    status:
+      code: 200
+      message: OK
+version: 1
diff --git a/tests/test_inference_async_client.py b/tests/test_inference_async_client.py
index c3416c8c..9679a9ac 100644
--- a/tests/test_inference_async_client.py
+++ b/tests/test_inference_async_client.py
@@ -33,15 +33,16 @@ import huggingface_hub.inference._common
 from huggingface_hub import (
     AsyncInferenceClient,
     ChatCompletionOutput,
-    ChatCompletionOutputChoice,
-    ChatCompletionOutputChoiceMessage,
+    ChatCompletionOutputComplete,
+    ChatCompletionOutputMessage,
+    ChatCompletionOutputUsage,
     ChatCompletionStreamOutput,
     InferenceClient,
     InferenceTimeoutError,
-    TextGenerationPrefillToken,
+    TextGenerationOutputPrefillToken,
 )
 from huggingface_hub.inference._common import ValidationError as TextGenerationValidationError
-from huggingface_hub.inference._common import _is_tgi_server
+from huggingface_hub.inference._common import _get_unsupported_text_generation_kwargs
 
 from .test_inference_client import CHAT_COMPLETE_NON_TGI_MODEL, CHAT_COMPLETION_MESSAGES, CHAT_COMPLETION_MODEL
 from .testing_utils import with_production_testing
@@ -49,7 +50,7 @@ from .testing_utils import with_production_testing
 
 @pytest.fixture(autouse=True)
 def patch_non_tgi_server(monkeypatch: pytest.MonkeyPatch):
-    monkeypatch.setattr(huggingface_hub.inference._common, "_NON_TGI_SERVERS", set())
+    monkeypatch.setattr(huggingface_hub.inference._common, "_UNSUPPORTED_TEXT_GENERATION_KWARGS", {})
 
 
 @pytest.fixture
@@ -74,7 +75,7 @@ async def test_async_generate_with_details(tgi_client: AsyncInferenceClient) ->
     assert response.details.generated_tokens == 1
     assert response.details.seed is None
     assert len(response.details.prefill) == 1
-    assert response.details.prefill[0] == TextGenerationPrefillToken(id=0, text="<pad>", logprob=None)
+    assert response.details.prefill[0] == TextGenerationOutputPrefillToken(id=0, text="<pad>", logprob=None)
     assert len(response.details.tokens) == 1
     assert response.details.tokens[0].id == 3
     assert response.details.tokens[0].text == " "
@@ -106,7 +107,7 @@ async def test_async_generate_validation_error(tgi_client: AsyncInferenceClient)
 async def test_async_generate_non_tgi_endpoint(tgi_client: AsyncInferenceClient) -> None:
     text = await tgi_client.text_generation("0 1 2", model="gpt2", max_new_tokens=10)
     assert text == " 3 4 5 6 7 8 9 10 11 12"
-    assert not _is_tgi_server("gpt2")
+    assert _get_unsupported_text_generation_kwargs("gpt2") == ["details", "stop", "watermark", "decoder_input_details"]
 
     # Watermark is ignored (+ warning)
     with pytest.warns(UserWarning):
@@ -160,11 +161,16 @@ async def test_async_chat_completion_no_stream() -> None:
     output = await async_client.chat_completion(CHAT_COMPLETION_MESSAGES, max_tokens=10)
     assert isinstance(output.created, int)
     assert output == ChatCompletionOutput(
+        id="",
+        model="HuggingFaceH4/zephyr-7b-beta",
+        object="text_completion",
+        system_fingerprint="1.4.3-sha-e6bb3ff",
+        usage=ChatCompletionOutputUsage(completion_tokens=10, prompt_tokens=47, total_tokens=57),
         choices=[
-            ChatCompletionOutputChoice(
+            ChatCompletionOutputComplete(
                 finish_reason="length",
                 index=0,
-                message=ChatCompletionOutputChoiceMessage(
+                message=ChatCompletionOutputMessage(
                     content="Deep learning is a subfield of machine learning that",
                     role="assistant",
                 ),
@@ -182,11 +188,16 @@ async def test_async_chat_completion_not_tgi_no_stream() -> None:
     output = await async_client.chat_completion(CHAT_COMPLETION_MESSAGES, max_tokens=10)
     assert isinstance(output.created, int)
     assert output == ChatCompletionOutput(
+        id="dummy",
+        model="dummy",
+        object="dummy",
+        system_fingerprint="dummy",
+        usage=None,
         choices=[
-            ChatCompletionOutputChoice(
+            ChatCompletionOutputComplete(
                 finish_reason="unk",  # Non-TGI => cannot know the finish reason
                 index=0,
-                message=ChatCompletionOutputChoiceMessage(
+                message=ChatCompletionOutputMessage(
                     content="Deep learning is a thing.",
                     role="assistant",
                 ),
diff --git a/tests/test_inference_client.py b/tests/test_inference_client.py
index 63789cdc..78e9435c 100644
--- a/tests/test_inference_client.py
+++ b/tests/test_inference_client.py
@@ -24,8 +24,8 @@ from PIL import Image
 from huggingface_hub import (
     AutomaticSpeechRecognitionOutput,
     ChatCompletionOutput,
-    ChatCompletionOutputChoice,
-    ChatCompletionOutputChoiceMessage,
+    ChatCompletionOutputComplete,
+    ChatCompletionOutputMessage,
     ChatCompletionStreamOutput,
     DocumentQuestionAnsweringOutputElement,
     FillMaskOutputElement,
@@ -82,6 +82,67 @@ CHAT_COMPLETION_MESSAGES = [
 ]
 CHAT_COMPLETE_NON_TGI_MODEL = "microsoft/DialoGPT-small"
 
+CHAT_COMPLETION_TOOL_INSTRUCTIONS = [
+    {
+        "role": "system",
+        "content": "Don't make assumptions about what values to plug into functions. Ask for clarification if a user request is ambiguous.",
+    },
+    {
+        "role": "user",
+        "content": "What's the weather like the next 3 days in San Francisco, CA?",
+    },
+]
+CHAT_COMPLETION_TOOLS = [  # 1 tool to get current weather, 1 to get N-day weather forecast
+    {
+        "type": "function",
+        "function": {
+            "name": "get_current_weather",
+            "description": "Get the current weather",
+            "parameters": {
+                "type": "object",
+                "properties": {
+                    "location": {
+                        "type": "string",
+                        "description": "The city and state, e.g. San Francisco, CA",
+                    },
+                    "format": {
+                        "type": "string",
+                        "enum": ["celsius", "fahrenheit"],
+                        "description": "The temperature unit to use. Infer this from the users location.",
+                    },
+                },
+                "required": ["location", "format"],
+            },
+        },
+    },
+    {
+        "type": "function",
+        "function": {
+            "name": "get_n_day_weather_forecast",
+            "description": "Get an N-day weather forecast",
+            "parameters": {
+                "type": "object",
+                "properties": {
+                    "location": {
+                        "type": "string",
+                        "description": "The city and state, e.g. San Francisco, CA",
+                    },
+                    "format": {
+                        "type": "string",
+                        "enum": ["celsius", "fahrenheit"],
+                        "description": "The temperature unit to use. Infer this from the users location.",
+                    },
+                    "num_days": {
+                        "type": "integer",
+                        "description": "The number of days to forecast",
+                    },
+                },
+                "required": ["location", "format", "num_days"],
+            },
+        },
+    },
+]
+
 
 class InferenceClientTest(unittest.TestCase):
     @classmethod
@@ -148,10 +209,10 @@ class InferenceClientVCRTest(InferenceClientTest):
         assert isinstance(output, ChatCompletionOutput)
         assert output.created < time.time()
         assert output.choices == [
-            ChatCompletionOutputChoice(
+            ChatCompletionOutputComplete(
                 finish_reason="length",
                 index=0,
-                message=ChatCompletionOutputChoiceMessage(
+                message=ChatCompletionOutputMessage(
                     content="Deep learning is a subfield of machine learning that focuses on training artificial neural networks with multiple layers of",
                     role="assistant",
                 ),
@@ -201,11 +262,16 @@ class InferenceClientVCRTest(InferenceClientTest):
             max_tokens=20,
         )
         assert output == ChatCompletionOutput(
+            id="dummy",
+            model="dummy",
+            object="dummy",
+            system_fingerprint="dummy",
+            usage=None,
             choices=[
-                ChatCompletionOutputChoice(
+                ChatCompletionOutputComplete(
                     finish_reason="unk",  # <- specific to models served with transformers (not possible to get details)
                     index=0,
-                    message=ChatCompletionOutputChoiceMessage(
+                    message=ChatCompletionOutputMessage(
                         content="Deep learning is a thing.",
                         role="assistant",
                     ),
@@ -214,6 +280,52 @@ class InferenceClientVCRTest(InferenceClientTest):
             created=output.created,
         )
 
+    def test_chat_completion_with_tool(self) -> None:
+        response = self.client.chat_completion(
+            model="meta-llama/Meta-Llama-3-70B-Instruct",
+            messages=CHAT_COMPLETION_TOOL_INSTRUCTIONS,
+            tools=CHAT_COMPLETION_TOOLS,
+            tool_choice="auto",
+            max_tokens=500,
+        )
+        output = response.choices[0]
+
+        # Single message before EOS
+        assert output.finish_reason == "eos_token"
+        assert output.index == 0
+        assert output.message.role == "assistant"
+
+        # No content but a tool call
+        assert output.message.content is None
+        assert len(output.message.tool_calls) == 1
+
+        # Tool
+        tool_call = output.message.tool_calls[0]
+        assert tool_call.type == "function"
+        assert tool_call.function.name == "get_n_day_weather_forecast"
+        assert tool_call.function.arguments == {
+            "format": "fahrenheit",
+            "location": "San Francisco, CA",
+            "num_days": 3,
+        }
+
+        # Now, test with tool_choice="get_current_weather"
+        response = self.client.chat_completion(
+            model="meta-llama/Meta-Llama-3-70B-Instruct",
+            messages=CHAT_COMPLETION_TOOL_INSTRUCTIONS,
+            tools=CHAT_COMPLETION_TOOLS,
+            tool_choice="get_current_weather",
+            max_tokens=500,
+        )
+        output = response.choices[0]
+        tool_call = output.message.tool_calls[0]
+        assert tool_call.function.name == "get_current_weather"
+        # No need for 'num_days' with this tool
+        assert tool_call.function.arguments == {
+            "format": "fahrenheit",
+            "location": "San Francisco, CA",
+        }
+
     @expect_deprecation("InferenceClient.conversational")
     def test_conversational(self) -> None:
         output = self.client.conversational("Hi, who are you?")
diff --git a/tests/test_inference_text_generation.py b/tests/test_inference_text_generation.py
index c2fcd678..8a552a1b 100644
--- a/tests/test_inference_text_generation.py
+++ b/tests/test_inference_text_generation.py
@@ -2,6 +2,7 @@
 # and https://github.com/huggingface/text-generation-inference/tree/main/clients/python)
 #
 # See './src/huggingface_hub/inference/_text_generation.py' for details.
+import json
 import unittest
 from typing import Dict
 from unittest.mock import MagicMock, patch
@@ -9,9 +10,9 @@ from unittest.mock import MagicMock, patch
 import pytest
 from requests import HTTPError
 
-from huggingface_hub import InferenceClient, TextGenerationPrefillToken
+from huggingface_hub import InferenceClient, TextGenerationOutputPrefillToken
 from huggingface_hub.inference._common import (
-    _NON_TGI_SERVERS,
+    _UNSUPPORTED_TEXT_GENERATION_KWARGS,
     GenerationError,
     IncompleteGenerationError,
     OverloadedError,
@@ -51,7 +52,7 @@ def _mocked_error(payload: Dict) -> MagicMock:
 
 
 @pytest.mark.vcr
-@patch.dict("huggingface_hub.inference._common._NON_TGI_SERVERS", {})
+@patch.dict("huggingface_hub.inference._common._UNSUPPORTED_TEXT_GENERATION_KWARGS", {})
 class TestTextGenerationClientVCR(unittest.TestCase):
     """Use VCR test to avoid making requests to the prod infra."""
 
@@ -72,7 +73,7 @@ class TestTextGenerationClientVCR(unittest.TestCase):
         assert response.details.generated_tokens == 1
         assert response.details.seed is None
         assert len(response.details.prefill) == 1
-        assert response.details.prefill[0] == TextGenerationPrefillToken(id=0, text="<pad>", logprob=None)
+        assert response.details.prefill[0] == TextGenerationOutputPrefillToken(id=0, text="<pad>", logprob=None)
         assert len(response.details.tokens) == 1
         assert response.details.tokens[0].id == 3
         assert response.details.tokens[0].text == " "
@@ -121,7 +122,7 @@ class TestTextGenerationClientVCR(unittest.TestCase):
     def test_generate_non_tgi_endpoint(self):
         text = self.client.text_generation("0 1 2", model="gpt2", max_new_tokens=10)
         self.assertEqual(text, " 3 4 5 6 7 8 9 10 11 12")
-        self.assertIn("gpt2", _NON_TGI_SERVERS)
+        self.assertIn("gpt2", _UNSUPPORTED_TEXT_GENERATION_KWARGS)
 
         # Watermark is ignored (+ warning)
         with self.assertWarns(UserWarning):
@@ -138,8 +139,35 @@ class TestTextGenerationClientVCR(unittest.TestCase):
 
     def test_generate_non_tgi_endpoint_regression_test(self):
         # Regression test for https://github.com/huggingface/huggingface_hub/issues/2135
-        with self.assertWarnsRegex(UserWarning, "Ignoring parameters .* 'return_full_text'"):
+        with self.assertWarnsRegex(UserWarning, "Ignoring following parameters: return_full_text"):
             text = self.client.text_generation(
-                prompt="How are you today?", max_new_tokens=20, model="google/flan-t5-large"
+                prompt="How are you today?", max_new_tokens=20, model="google/flan-t5-large", return_full_text=True
             )
         assert text == "I am at work"
+
+    def test_generate_with_grammar(self):
+        # Example taken from https://huggingface.co/docs/text-generation-inference/conceptual/guidance#the-grammar-parameter
+        response = self.client.text_generation(
+            prompt="I saw a puppy a cat and a raccoon during my bike ride in the park",
+            max_new_tokens=100,
+            model="HuggingFaceH4/zephyr-orpo-141b-A35b-v0.1",
+            repetition_penalty=1.3,
+            grammar={
+                "type": "json",
+                "value": {
+                    "properties": {
+                        "location": {"type": "string"},
+                        "activity": {"type": "string"},
+                        "animals_seen": {"type": "integer", "minimum": 1, "maximum": 5},
+                        "animals": {"type": "array", "items": {"type": "string"}},
+                    },
+                    "required": ["location", "activity", "animals_seen", "animals"],
+                },
+            },
+        )
+        assert json.loads(response) == {
+            "activity": "biking",
+            "animals": [],
+            "animals_seen": 3,
+            "location": "park",
+        }