Repository: huggingface/huggingface_hub
Pull request #2036 (2024-02-20 17:50:09)

Generate inference types + start using output types
cc @SBrandeis @julien-c for visibility.

This PR adds scripts to generate inference types + start to use the output types in the InferenceClient.

**The broader goal is to use all the spec-ed types in the `InferenceClient` but also in `transformers` and `diffusers` pipelines.** 

**How it works:**
- generating script in huggingface.js/tasks => done separately
- [utils/generate_inference_types.py](https://github.com/huggingface/huggingface_hub/pull/2036/files#diff-00d1e691494e271ddeb890fe99c7e676758667ff83a5dd88cf0b0679743d8468) => clean Python generated code + generate imports + generate package reference
- all generated classes inherit from [src/huggingface_hub/inference/_generated/types/base.py](https://github.com/huggingface/huggingface_hub/pull/2036/files#diff-911e1fb94189e3763888d76686c868aa0bfeb598195c7be00ed6b521d5004c0c) 
  - dataclass + inherit from dict for backward compatibility => goal is to make it dataclass-only in the future (1.0?)
  - adds parsing methods to avoid pydantic
  - all fields are optional E.g. don't raise if server "forget" a required field
  - extra fields are set as attributes for future compatibility
- output types are now used in [src/huggingface_hub/inference/_client.py](https://github.com/huggingface/huggingface_hub/pull/2036/files#diff-9f7e6ce951acea8a760297ca71a4a8995e42f66304f02ebf54dac80a051815ae)

**Docs:**
- [Inference types](https://moon-ci-docs.huggingface.co/docs/huggingface_hub/pr_2036/en/package_reference/inference_types)
- [Inference client](https://moon-ci-docs.huggingface.co/docs/huggingface_hub/pr_2036/en/package_reference/inference_client)

**TODO:**
- [x] add types to inference package reference in docs
- [x] how to handle parsing audio or images in output? currently done "manually", e.g. not from the json specs
- [x] fix tests
- [x] fix type annotations in `_client.py`...
- [x] make dataclasses first class citizen in `huggingface_hub`?
- [ ] better auto-generated docstrings => following HF docs format => will be done later

**Breaking changes in outputs :broken_heart:**:
- `automatic_speech_recognition`: str => `AutomaticSpeechRecognitionOutput`
- `summarization`: str => `SummarizationOutput`
- `translation`: str => `TranslationOutput`

**Left out of scope for this PR:**
- text_generation => requires more typing for `stream=True`
- deal with `conversational` => will be deprecated/removed soon
- use input parameters generated from specs

codecov[bot]: ## [Codecov](https://app.codecov.io/gh/huggingface/huggingface_hub/pull/2036?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface) Report
Attention: `25 lines` in your changes are missing coverage. Please review.
> Comparison is base [(`e05fb73`)](https://app.codecov.io/gh/huggingface/huggingface_hub/commit/e05fb737a08bfa5c6b9b40998b77c0ef7548a8f8?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface) 82.22% compared to head [(`8c83911`)](https://app.codecov.io/gh/huggingface/huggingface_hub/pull/2036?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface) 83.80%.

| [Files](https://app.codecov.io/gh/huggingface/huggingface_hub/pull/2036?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface) | Patch % | Lines |
|---|---|---|
| [...gingface\_hub/inference/\_generated/\_async\_client.py](https://app.codecov.io/gh/huggingface/huggingface_hub/pull/2036?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface#diff-c3JjL2h1Z2dpbmdmYWNlX2h1Yi9pbmZlcmVuY2UvX2dlbmVyYXRlZC9fYXN5bmNfY2xpZW50LnB5) | 25.00% | [15 Missing :warning: ](https://app.codecov.io/gh/huggingface/huggingface_hub/pull/2036?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface) |
| [...huggingface\_hub/inference/\_generated/types/base.py](https://app.codecov.io/gh/huggingface/huggingface_hub/pull/2036?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface#diff-c3JjL2h1Z2dpbmdmYWNlX2h1Yi9pbmZlcmVuY2UvX2dlbmVyYXRlZC90eXBlcy9iYXNlLnB5) | 81.81% | [10 Missing :warning: ](https://app.codecov.io/gh/huggingface/huggingface_hub/pull/2036?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface) |

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##             main    #2036      +/-   ##
==========================================
+ Coverage   82.22%   83.80%   +1.58%     
==========================================
  Files          66       96      +30     
  Lines        8309     9282     +973     
==========================================
+ Hits         6832     7779     +947     
- Misses       1477     1503      +26     
```



</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/huggingface/huggingface_hub/pull/2036?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=huggingface).

Wauplin: This PR is now ready for review :)  :tada:
I've updated the PR description accordingly. This PR will introduce breaking changes for some tasks but hopefully _not too problematic_. I'm planning to make a release of `huggingface_hub` early next week but no rush to merge this PR. It can wait the next one which would allow to implement a few more TODOs before shipping.  
SBrandeis: Very cool that the inference types are used in the Inference client right away!
Wauplin: Let's merge this PR now that everything's addressed! I have listed remaining todos in a follow-up issue: https://github.com/huggingface/huggingface_hub/issues/2063.

Patch:
diff --git a/.github/workflows/python-quality.yml b/.github/workflows/python-quality.yml
index 037cebf7..3d400487 100644
--- a/.github/workflows/python-quality.yml
+++ b/.github/workflows/python-quality.yml
@@ -32,6 +32,7 @@ jobs:
       - run: python utils/check_contrib_list.py
       - run: python utils/check_static_imports.py
       - run: python utils/generate_async_inference_client.py
+      - run: python utils/generate_inference_types.py
 
       # Run type checking at least on huggingface_hub root file to check all modules
       # that can be lazy-loaded actually exist.
diff --git a/Makefile b/Makefile
index 4a8f6397..ad2a0520 100644
--- a/Makefile
+++ b/Makefile
@@ -19,6 +19,12 @@ style:
 	python utils/check_static_imports.py --update
 	python utils/generate_async_inference_client.py --update
 
+inference_types_check:
+	python utils/generate_inference_types.py
+
+inference_types_update:
+	python utils/generate_inference_types.py --update
+
 repocard:
 	python utils/push_repocard_examples.py
 
diff --git a/docs/source/en/_toctree.yml b/docs/source/en/_toctree.yml
index 50326714..36efe4ca 100644
--- a/docs/source/en/_toctree.yml
+++ b/docs/source/en/_toctree.yml
@@ -60,6 +60,8 @@
       title: Downloading files
     - local: package_reference/mixins
       title: Mixins & serialization methods
+    - local: package_reference/inference_types
+      title: Inference Types
     - local: package_reference/inference_client
       title: Inference Client
     - local: package_reference/inference_endpoints
diff --git a/docs/source/en/package_reference/inference_client.md b/docs/source/en/package_reference/inference_client.md
index bba3a4f8..39b67c45 100644
--- a/docs/source/en/package_reference/inference_client.md
+++ b/docs/source/en/package_reference/inference_client.md
@@ -41,10 +41,6 @@ pip install --upgrade huggingface_hub[inference]
 
 For most tasks, the return value has a built-in type (string, list, image...). Here is a list for the more complex types.
 
-### ClassificationOutput
-
-[[autodoc]] huggingface_hub.inference._types.ClassificationOutput
-
 ### ConversationalOutputConversation
 
 [[autodoc]] huggingface_hub.inference._types.ConversationalOutputConversation
@@ -53,18 +49,10 @@ For most tasks, the return value has a built-in type (string, list, image...). H
 
 [[autodoc]] huggingface_hub.inference._types.ConversationalOutput
 
-### ImageSegmentationOutput
-
-[[autodoc]] huggingface_hub.inference._types.ImageSegmentationOutput
-
 ### ModelStatus
 
 [[autodoc]] huggingface_hub.inference._common.ModelStatus
 
-### TokenClassificationOutput
-
-[[autodoc]] huggingface_hub.inference._types.TokenClassificationOutput
-
 ### Text generation types
 
 [`~InferenceClient.text_generation`] task has a greater support than other tasks in `InferenceClient`. In
diff --git a/docs/source/en/package_reference/inference_types.md b/docs/source/en/package_reference/inference_types.md
new file mode 100644
index 00000000..9b0ed274
--- /dev/null
+++ b/docs/source/en/package_reference/inference_types.md
@@ -0,0 +1,334 @@
+
+<!--⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be
+rendered properly in your Markdown viewer.
+-->
+<!--⚠️ Note that this file is auto-generated by `utils/generate_inference_types.py`. Do not modify it manually.-->
+
+
+# Inference types
+
+This page lists the types (e.g. dataclasses) available for each task supported on the Hugging Face Hub.
+Each task is specified using a JSON schema, and the types are generated from these schemas - with some customization
+due to Python requirements.
+Visit [@huggingface.js/tasks](https://github.com/huggingface/huggingface.js/tree/main/packages/tasks/src/tasks)
+to find the JSON schemas for each task.
+
+This part of the lib is still under development and will be improved in future releases.
+
+
+
+## audio_classification
+
+[[autodoc]] huggingface_hub.AudioClassificationInput
+
+[[autodoc]] huggingface_hub.AudioClassificationOutputElement
+
+[[autodoc]] huggingface_hub.AudioClassificationParameters
+
+
+
+## audio_to_audio
+
+[[autodoc]] huggingface_hub.AudioToAudioInput
+
+[[autodoc]] huggingface_hub.AudioToAudioOutputElement
+
+
+
+## automatic_speech_recognition
+
+[[autodoc]] huggingface_hub.AutomaticSpeechRecognitionInput
+
+[[autodoc]] huggingface_hub.AutomaticSpeechRecognitionOutput
+
+[[autodoc]] huggingface_hub.AutomaticSpeechRecognitionOutputChunk
+
+[[autodoc]] huggingface_hub.AutomaticSpeechRecognitionParameters
+
+[[autodoc]] huggingface_hub.GenerationParameters
+
+
+
+## depth_estimation
+
+[[autodoc]] huggingface_hub.DepthEstimationInput
+
+[[autodoc]] huggingface_hub.DepthEstimationOutput
+
+
+
+## document_question_answering
+
+[[autodoc]] huggingface_hub.DocumentQuestionAnsweringInput
+
+[[autodoc]] huggingface_hub.DocumentQuestionAnsweringInputData
+
+[[autodoc]] huggingface_hub.DocumentQuestionAnsweringOutputElement
+
+[[autodoc]] huggingface_hub.DocumentQuestionAnsweringParameters
+
+
+
+## feature_extraction
+
+[[autodoc]] huggingface_hub.FeatureExtractionInput
+
+
+
+## fill_mask
+
+[[autodoc]] huggingface_hub.FillMaskInput
+
+[[autodoc]] huggingface_hub.FillMaskOutputElement
+
+[[autodoc]] huggingface_hub.FillMaskParameters
+
+
+
+## image_classification
+
+[[autodoc]] huggingface_hub.ImageClassificationInput
+
+[[autodoc]] huggingface_hub.ImageClassificationOutputElement
+
+[[autodoc]] huggingface_hub.ImageClassificationParameters
+
+
+
+## image_segmentation
+
+[[autodoc]] huggingface_hub.ImageSegmentationInput
+
+[[autodoc]] huggingface_hub.ImageSegmentationOutputElement
+
+[[autodoc]] huggingface_hub.ImageSegmentationParameters
+
+
+
+## image_to_image
+
+[[autodoc]] huggingface_hub.ImageToImageInput
+
+[[autodoc]] huggingface_hub.ImageToImageOutput
+
+[[autodoc]] huggingface_hub.ImageToImageParameters
+
+[[autodoc]] huggingface_hub.TargetSize
+
+
+
+## image_to_text
+
+[[autodoc]] huggingface_hub.GenerationParameters
+
+[[autodoc]] huggingface_hub.ImageToTextInput
+
+[[autodoc]] huggingface_hub.ImageToTextOutput
+
+[[autodoc]] huggingface_hub.ImageToTextParameters
+
+
+
+## object_detection
+
+[[autodoc]] huggingface_hub.BoundingBox
+
+[[autodoc]] huggingface_hub.ObjectDetectionInput
+
+[[autodoc]] huggingface_hub.ObjectDetectionOutputElement
+
+[[autodoc]] huggingface_hub.ObjectDetectionParameters
+
+
+
+## question_answering
+
+[[autodoc]] huggingface_hub.QuestionAnsweringInput
+
+[[autodoc]] huggingface_hub.QuestionAnsweringInputData
+
+[[autodoc]] huggingface_hub.QuestionAnsweringOutputElement
+
+[[autodoc]] huggingface_hub.QuestionAnsweringParameters
+
+
+
+## sentence_similarity
+
+[[autodoc]] huggingface_hub.SentenceSimilarityInput
+
+[[autodoc]] huggingface_hub.SentenceSimilarityInputData
+
+
+
+## summarization
+
+[[autodoc]] huggingface_hub.SummarizationInput
+
+[[autodoc]] huggingface_hub.SummarizationOutput
+
+[[autodoc]] huggingface_hub.Text2TextGenerationParameters
+
+
+
+## table_question_answering
+
+[[autodoc]] huggingface_hub.TableQuestionAnsweringInput
+
+[[autodoc]] huggingface_hub.TableQuestionAnsweringInputData
+
+[[autodoc]] huggingface_hub.TableQuestionAnsweringOutputElement
+
+
+
+## text2text_generation
+
+[[autodoc]] huggingface_hub.Text2TextGenerationInput
+
+[[autodoc]] huggingface_hub.Text2TextGenerationOutput
+
+[[autodoc]] huggingface_hub.Text2TextGenerationParameters
+
+
+
+## text_classification
+
+[[autodoc]] huggingface_hub.TextClassificationInput
+
+[[autodoc]] huggingface_hub.TextClassificationOutputElement
+
+[[autodoc]] huggingface_hub.TextClassificationParameters
+
+
+
+## text_generation
+
+[[autodoc]] huggingface_hub.PrefillToken
+
+[[autodoc]] huggingface_hub.TextGenerationInput
+
+[[autodoc]] huggingface_hub.TextGenerationOutput
+
+[[autodoc]] huggingface_hub.TextGenerationOutputDetails
+
+[[autodoc]] huggingface_hub.TextGenerationParameters
+
+[[autodoc]] huggingface_hub.TextGenerationSequenceDetails
+
+[[autodoc]] huggingface_hub.Token
+
+
+
+## text_to_audio
+
+[[autodoc]] huggingface_hub.GenerationParameters
+
+[[autodoc]] huggingface_hub.TextToAudioInput
+
+[[autodoc]] huggingface_hub.TextToAudioOutput
+
+[[autodoc]] huggingface_hub.TextToAudioParameters
+
+
+
+## text_to_image
+
+[[autodoc]] huggingface_hub.TargetSize
+
+[[autodoc]] huggingface_hub.TextToImageInput
+
+[[autodoc]] huggingface_hub.TextToImageOutput
+
+[[autodoc]] huggingface_hub.TextToImageParameters
+
+
+
+## text_to_speech
+
+[[autodoc]] huggingface_hub.GenerationParameters
+
+[[autodoc]] huggingface_hub.TextToAudioParameters
+
+[[autodoc]] huggingface_hub.TextToSpeechInput
+
+[[autodoc]] huggingface_hub.TextToSpeechOutput
+
+
+
+## token_classification
+
+[[autodoc]] huggingface_hub.TokenClassificationInput
+
+[[autodoc]] huggingface_hub.TokenClassificationOutputElement
+
+[[autodoc]] huggingface_hub.TokenClassificationParameters
+
+
+
+## translation
+
+[[autodoc]] huggingface_hub.Text2TextGenerationParameters
+
+[[autodoc]] huggingface_hub.TranslationInput
+
+[[autodoc]] huggingface_hub.TranslationOutput
+
+
+
+## video_classification
+
+[[autodoc]] huggingface_hub.VideoClassificationInput
+
+[[autodoc]] huggingface_hub.VideoClassificationOutputElement
+
+[[autodoc]] huggingface_hub.VideoClassificationParameters
+
+
+
+## visual_question_answering
+
+[[autodoc]] huggingface_hub.VisualQuestionAnsweringInput
+
+[[autodoc]] huggingface_hub.VisualQuestionAnsweringInputData
+
+[[autodoc]] huggingface_hub.VisualQuestionAnsweringOutputElement
+
+[[autodoc]] huggingface_hub.VisualQuestionAnsweringParameters
+
+
+
+## zero_shot_classification
+
+[[autodoc]] huggingface_hub.ZeroShotClassificationInput
+
+[[autodoc]] huggingface_hub.ZeroShotClassificationInputData
+
+[[autodoc]] huggingface_hub.ZeroShotClassificationOutputElement
+
+[[autodoc]] huggingface_hub.ZeroShotClassificationParameters
+
+
+
+## zero_shot_image_classification
+
+[[autodoc]] huggingface_hub.ZeroShotImageClassificationInput
+
+[[autodoc]] huggingface_hub.ZeroShotImageClassificationInputData
+
+[[autodoc]] huggingface_hub.ZeroShotImageClassificationOutputElement
+
+[[autodoc]] huggingface_hub.ZeroShotImageClassificationParameters
+
+
+
+## zero_shot_object_detection
+
+[[autodoc]] huggingface_hub.BoundingBox
+
+[[autodoc]] huggingface_hub.ZeroShotObjectDetectionInput
+
+[[autodoc]] huggingface_hub.ZeroShotObjectDetectionInputData
+
+[[autodoc]] huggingface_hub.ZeroShotObjectDetectionOutputElement
+
+
diff --git a/src/huggingface_hub/__init__.py b/src/huggingface_hub/__init__.py
index e1cb5846..907049a7 100644
--- a/src/huggingface_hub/__init__.py
+++ b/src/huggingface_hub/__init__.py
@@ -259,6 +259,100 @@ _SUBMOD_ATTRS = {
     "inference._generated._async_client": [
         "AsyncInferenceClient",
     ],
+    "inference._generated.types": [
+        "AudioClassificationInput",
+        "AudioClassificationOutputElement",
+        "AudioClassificationParameters",
+        "AudioToAudioInput",
+        "AudioToAudioOutputElement",
+        "AutomaticSpeechRecognitionInput",
+        "AutomaticSpeechRecognitionOutput",
+        "AutomaticSpeechRecognitionOutputChunk",
+        "AutomaticSpeechRecognitionParameters",
+        "BoundingBox",
+        "DepthEstimationInput",
+        "DepthEstimationOutput",
+        "DocumentQuestionAnsweringInput",
+        "DocumentQuestionAnsweringInputData",
+        "DocumentQuestionAnsweringOutputElement",
+        "DocumentQuestionAnsweringParameters",
+        "FeatureExtractionInput",
+        "FillMaskInput",
+        "FillMaskOutputElement",
+        "FillMaskParameters",
+        "GenerationParameters",
+        "ImageClassificationInput",
+        "ImageClassificationOutputElement",
+        "ImageClassificationParameters",
+        "ImageSegmentationInput",
+        "ImageSegmentationOutputElement",
+        "ImageSegmentationParameters",
+        "ImageToImageInput",
+        "ImageToImageOutput",
+        "ImageToImageParameters",
+        "ImageToTextInput",
+        "ImageToTextOutput",
+        "ImageToTextParameters",
+        "ObjectDetectionInput",
+        "ObjectDetectionOutputElement",
+        "ObjectDetectionParameters",
+        "PrefillToken",
+        "QuestionAnsweringInput",
+        "QuestionAnsweringInputData",
+        "QuestionAnsweringOutputElement",
+        "QuestionAnsweringParameters",
+        "SentenceSimilarityInput",
+        "SentenceSimilarityInputData",
+        "SummarizationInput",
+        "SummarizationOutput",
+        "TableQuestionAnsweringInput",
+        "TableQuestionAnsweringInputData",
+        "TableQuestionAnsweringOutputElement",
+        "TargetSize",
+        "Text2TextGenerationInput",
+        "Text2TextGenerationOutput",
+        "Text2TextGenerationParameters",
+        "TextClassificationInput",
+        "TextClassificationOutputElement",
+        "TextClassificationParameters",
+        "TextGenerationInput",
+        "TextGenerationOutput",
+        "TextGenerationOutputDetails",
+        "TextGenerationParameters",
+        "TextGenerationSequenceDetails",
+        "TextToAudioInput",
+        "TextToAudioOutput",
+        "TextToAudioParameters",
+        "TextToImageInput",
+        "TextToImageOutput",
+        "TextToImageParameters",
+        "TextToSpeechInput",
+        "TextToSpeechOutput",
+        "Token",
+        "TokenClassificationInput",
+        "TokenClassificationOutputElement",
+        "TokenClassificationParameters",
+        "TranslationInput",
+        "TranslationOutput",
+        "VideoClassificationInput",
+        "VideoClassificationOutputElement",
+        "VideoClassificationParameters",
+        "VisualQuestionAnsweringInput",
+        "VisualQuestionAnsweringInputData",
+        "VisualQuestionAnsweringOutputElement",
+        "VisualQuestionAnsweringParameters",
+        "ZeroShotClassificationInput",
+        "ZeroShotClassificationInputData",
+        "ZeroShotClassificationOutputElement",
+        "ZeroShotClassificationParameters",
+        "ZeroShotImageClassificationInput",
+        "ZeroShotImageClassificationInputData",
+        "ZeroShotImageClassificationOutputElement",
+        "ZeroShotImageClassificationParameters",
+        "ZeroShotObjectDetectionInput",
+        "ZeroShotObjectDetectionInputData",
+        "ZeroShotObjectDetectionOutputElement",
+    ],
     "inference_api": [
         "InferenceApi",
     ],
@@ -613,6 +707,100 @@ if TYPE_CHECKING:  # pragma: no cover
         InferenceTimeoutError,  # noqa: F401
     )
     from .inference._generated._async_client import AsyncInferenceClient  # noqa: F401
+    from .inference._generated.types import (
+        AudioClassificationInput,  # noqa: F401
+        AudioClassificationOutputElement,  # noqa: F401
+        AudioClassificationParameters,  # noqa: F401
+        AudioToAudioInput,  # noqa: F401
+        AudioToAudioOutputElement,  # noqa: F401
+        AutomaticSpeechRecognitionInput,  # noqa: F401
+        AutomaticSpeechRecognitionOutput,  # noqa: F401
+        AutomaticSpeechRecognitionOutputChunk,  # noqa: F401
+        AutomaticSpeechRecognitionParameters,  # noqa: F401
+        BoundingBox,  # noqa: F401
+        DepthEstimationInput,  # noqa: F401
+        DepthEstimationOutput,  # noqa: F401
+        DocumentQuestionAnsweringInput,  # noqa: F401
+        DocumentQuestionAnsweringInputData,  # noqa: F401
+        DocumentQuestionAnsweringOutputElement,  # noqa: F401
+        DocumentQuestionAnsweringParameters,  # noqa: F401
+        FeatureExtractionInput,  # noqa: F401
+        FillMaskInput,  # noqa: F401
+        FillMaskOutputElement,  # noqa: F401
+        FillMaskParameters,  # noqa: F401
+        GenerationParameters,  # noqa: F401
+        ImageClassificationInput,  # noqa: F401
+        ImageClassificationOutputElement,  # noqa: F401
+        ImageClassificationParameters,  # noqa: F401
+        ImageSegmentationInput,  # noqa: F401
+        ImageSegmentationOutputElement,  # noqa: F401
+        ImageSegmentationParameters,  # noqa: F401
+        ImageToImageInput,  # noqa: F401
+        ImageToImageOutput,  # noqa: F401
+        ImageToImageParameters,  # noqa: F401
+        ImageToTextInput,  # noqa: F401
+        ImageToTextOutput,  # noqa: F401
+        ImageToTextParameters,  # noqa: F401
+        ObjectDetectionInput,  # noqa: F401
+        ObjectDetectionOutputElement,  # noqa: F401
+        ObjectDetectionParameters,  # noqa: F401
+        PrefillToken,  # noqa: F401
+        QuestionAnsweringInput,  # noqa: F401
+        QuestionAnsweringInputData,  # noqa: F401
+        QuestionAnsweringOutputElement,  # noqa: F401
+        QuestionAnsweringParameters,  # noqa: F401
+        SentenceSimilarityInput,  # noqa: F401
+        SentenceSimilarityInputData,  # noqa: F401
+        SummarizationInput,  # noqa: F401
+        SummarizationOutput,  # noqa: F401
+        TableQuestionAnsweringInput,  # noqa: F401
+        TableQuestionAnsweringInputData,  # noqa: F401
+        TableQuestionAnsweringOutputElement,  # noqa: F401
+        TargetSize,  # noqa: F401
+        Text2TextGenerationInput,  # noqa: F401
+        Text2TextGenerationOutput,  # noqa: F401
+        Text2TextGenerationParameters,  # noqa: F401
+        TextClassificationInput,  # noqa: F401
+        TextClassificationOutputElement,  # noqa: F401
+        TextClassificationParameters,  # noqa: F401
+        TextGenerationInput,  # noqa: F401
+        TextGenerationOutput,  # noqa: F401
+        TextGenerationOutputDetails,  # noqa: F401
+        TextGenerationParameters,  # noqa: F401
+        TextGenerationSequenceDetails,  # noqa: F401
+        TextToAudioInput,  # noqa: F401
+        TextToAudioOutput,  # noqa: F401
+        TextToAudioParameters,  # noqa: F401
+        TextToImageInput,  # noqa: F401
+        TextToImageOutput,  # noqa: F401
+        TextToImageParameters,  # noqa: F401
+        TextToSpeechInput,  # noqa: F401
+        TextToSpeechOutput,  # noqa: F401
+        Token,  # noqa: F401
+        TokenClassificationInput,  # noqa: F401
+        TokenClassificationOutputElement,  # noqa: F401
+        TokenClassificationParameters,  # noqa: F401
+        TranslationInput,  # noqa: F401
+        TranslationOutput,  # noqa: F401
+        VideoClassificationInput,  # noqa: F401
+        VideoClassificationOutputElement,  # noqa: F401
+        VideoClassificationParameters,  # noqa: F401
+        VisualQuestionAnsweringInput,  # noqa: F401
+        VisualQuestionAnsweringInputData,  # noqa: F401
+        VisualQuestionAnsweringOutputElement,  # noqa: F401
+        VisualQuestionAnsweringParameters,  # noqa: F401
+        ZeroShotClassificationInput,  # noqa: F401
+        ZeroShotClassificationInputData,  # noqa: F401
+        ZeroShotClassificationOutputElement,  # noqa: F401
+        ZeroShotClassificationParameters,  # noqa: F401
+        ZeroShotImageClassificationInput,  # noqa: F401
+        ZeroShotImageClassificationInputData,  # noqa: F401
+        ZeroShotImageClassificationOutputElement,  # noqa: F401
+        ZeroShotImageClassificationParameters,  # noqa: F401
+        ZeroShotObjectDetectionInput,  # noqa: F401
+        ZeroShotObjectDetectionInputData,  # noqa: F401
+        ZeroShotObjectDetectionOutputElement,  # noqa: F401
+    )
     from .inference_api import InferenceApi  # noqa: F401
     from .keras_mixin import (
         KerasModelHubMixin,  # noqa: F401
diff --git a/src/huggingface_hub/inference/_client.py b/src/huggingface_hub/inference/_client.py
index f62492d0..4871fea6 100644
--- a/src/huggingface_hub/inference/_client.py
+++ b/src/huggingface_hub/inference/_client.py
@@ -23,7 +23,6 @@
 #    https://github.com/huggingface/unity-api#tasks
 #
 # Some TODO:
-# - validate inputs/options/parameters? with Pydantic for instance? or only optionally?
 # - add all tasks
 #
 # NOTE: the philosophy of this client is "let's make it as easy as possible to use it, even if less optimized". Some
@@ -71,6 +70,26 @@ from huggingface_hub.inference._common import (
     _set_as_non_tgi,
     _stream_text_generation_response,
 )
+from huggingface_hub.inference._generated.types import (
+    AudioClassificationOutputElement,
+    AudioToAudioOutputElement,
+    AutomaticSpeechRecognitionOutput,
+    DocumentQuestionAnsweringOutputElement,
+    FillMaskOutputElement,
+    ImageClassificationOutputElement,
+    ImageSegmentationOutputElement,
+    ImageToTextOutput,
+    ObjectDetectionOutputElement,
+    QuestionAnsweringOutputElement,
+    SummarizationOutput,
+    TableQuestionAnsweringOutputElement,
+    TextClassificationOutputElement,
+    TokenClassificationOutputElement,
+    TranslationOutput,
+    VisualQuestionAnsweringOutputElement,
+    ZeroShotClassificationOutputElement,
+    ZeroShotImageClassificationOutputElement,
+)
 from huggingface_hub.inference._text_generation import (
     TextGenerationParameters,
     TextGenerationRequest,
@@ -79,15 +98,7 @@ from huggingface_hub.inference._text_generation import (
     raise_text_generation_error,
 )
 from huggingface_hub.inference._types import (
-    AudioToAudioOutput,
-    ClassificationOutput,
-    ConversationalOutput,
-    FillMaskOutput,
-    ImageSegmentationOutput,
-    ObjectDetectionOutput,
-    QuestionAnsweringOutput,
-    TableQuestionAnsweringOutput,
-    TokenClassificationOutput,
+    ConversationalOutput,  # soon to be removed
 )
 from huggingface_hub.utils import (
     BadRequestError,
@@ -268,7 +279,7 @@ class InferenceClient:
         audio: ContentT,
         *,
         model: Optional[str] = None,
-    ) -> List[ClassificationOutput]:
+    ) -> List[AudioClassificationOutputElement]:
         """
         Perform audio classification on the provided audio content.
 
@@ -282,7 +293,7 @@ class InferenceClient:
                 audio classification will be used.
 
         Returns:
-            `List[Dict]`: The classification output containing the predicted label and its confidence.
+            `List[AudioClassificationOutputElement]`: List of [`AudioClassificationOutputElement`] items containing the predicted labels and their confidence.
 
         Raises:
             [`InferenceTimeoutError`]:
@@ -295,18 +306,22 @@ class InferenceClient:
         >>> from huggingface_hub import InferenceClient
         >>> client = InferenceClient()
         >>> client.audio_classification("audio.flac")
-        [{'score': 0.4976358711719513, 'label': 'hap'}, {'score': 0.3677836060523987, 'label': 'neu'},...]
+        [
+            AudioClassificationOutputElement(score=0.4976358711719513, label='hap'),
+            AudioClassificationOutputElement(score=0.3677836060523987, label='neu'),
+            ...
+        ]
         ```
         """
         response = self.post(data=audio, model=model, task="audio-classification")
-        return _bytes_to_list(response)
+        return AudioClassificationOutputElement.parse_obj_as_list(response)
 
     def audio_to_audio(
         self,
         audio: ContentT,
         *,
         model: Optional[str] = None,
-    ) -> List[AudioToAudioOutput]:
+    ) -> List[AudioToAudioOutputElement]:
         """
         Performs multiple tasks related to audio-to-audio depending on the model (eg: speech enhancement, source separation).
 
@@ -320,7 +335,7 @@ class InferenceClient:
                 audio_to_audio will be used.
 
         Returns:
-            `List[Dict]`: A list of dictionary where each index contains audios label, content-type, and audio content in blob.
+            `List[AudioToAudioOutputElement]`: A list of [`AudioToAudioOutputElement`] items containing audios label, content-type, and audio content in blob.
 
         Raises:
             `InferenceTimeoutError`:
@@ -335,13 +350,13 @@ class InferenceClient:
         >>> audio_output = client.audio_to_audio("audio.flac")
         >>> for i, item in enumerate(audio_output):
         >>>     with open(f"output_{i}.flac", "wb") as f:
-                    f.write(item["blob"])
+                    f.write(item.blob)
         ```
         """
         response = self.post(data=audio, model=model, task="audio-to-audio")
-        audio_output = _bytes_to_list(response)
+        audio_output = AudioToAudioOutputElement.parse_obj_as_list(response)
         for item in audio_output:
-            item["blob"] = base64.b64decode(item["blob"])
+            item.blob = base64.b64decode(item.blob)
         return audio_output
 
     def automatic_speech_recognition(
@@ -349,7 +364,7 @@ class InferenceClient:
         audio: ContentT,
         *,
         model: Optional[str] = None,
-    ) -> str:
+    ) -> AutomaticSpeechRecognitionOutput:
         """
         Perform automatic speech recognition (ASR or audio-to-text) on the given audio content.
 
@@ -361,7 +376,7 @@ class InferenceClient:
                 Inference Endpoint. If not provided, the default recommended model for ASR will be used.
 
         Returns:
-            str: The transcribed text.
+            [`AutomaticSpeechRecognitionOutput`]: An item containing the transcribed text and optionally the timestamp chunks.
 
         Raises:
             [`InferenceTimeoutError`]:
@@ -373,12 +388,12 @@ class InferenceClient:
         ```py
         >>> from huggingface_hub import InferenceClient
         >>> client = InferenceClient()
-        >>> client.automatic_speech_recognition("hello_world.flac")
+        >>> client.automatic_speech_recognition("hello_world.flac").text
         "hello world"
         ```
         """
         response = self.post(data=audio, model=model, task="automatic-speech-recognition")
-        return _bytes_to_dict(response)["text"]
+        return AutomaticSpeechRecognitionOutput.parse_obj_as_instance(response)
 
     def conversational(
         self,
@@ -441,57 +456,13 @@ class InferenceClient:
         response = self.post(json=payload, model=model, task="conversational")
         return _bytes_to_dict(response)  # type: ignore
 
-    def visual_question_answering(
-        self,
-        image: ContentT,
-        question: str,
-        *,
-        model: Optional[str] = None,
-    ) -> List[str]:
-        """
-        Answering open-ended questions based on an image.
-
-        Args:
-            image (`Union[str, Path, bytes, BinaryIO]`):
-                The input image for the context. It can be raw bytes, an image file, or a URL to an online image.
-            question (`str`):
-                Question to be answered.
-            model (`str`, *optional*):
-                The model to use for the visual question answering task. Can be a model ID hosted on the Hugging Face Hub or a URL to
-                a deployed Inference Endpoint. If not provided, the default recommended visual question answering model will be used.
-                Defaults to None.
-
-        Returns:
-            `List[Dict]`: a list of dictionaries containing the predicted label and associated probability.
-
-        Raises:
-            `InferenceTimeoutError`:
-                If the model is unavailable or the request times out.
-            `HTTPError`:
-                If the request fails with an HTTP error status code other than HTTP 503.
-
-        Example:
-        ```py
-        >>> from huggingface_hub import InferenceClient
-        >>> client = InferenceClient()
-        >>> client.visual_question_answering(
-        ...     image="https://huggingface.co/datasets/mishig/sample_images/resolve/main/tiger.jpg",
-        ...     question="What is the animal doing?"
-        ... )
-        [{'score': 0.778609573841095, 'answer': 'laying down'},{'score': 0.6957435607910156, 'answer': 'sitting'}, ...]
-        ```
-        """
-        payload: Dict[str, Any] = {"question": question, "image": _b64_encode(image)}
-        response = self.post(json=payload, model=model, task="visual-question-answering")
-        return _bytes_to_list(response)
-
     def document_question_answering(
         self,
         image: ContentT,
         question: str,
         *,
         model: Optional[str] = None,
-    ) -> List[QuestionAnsweringOutput]:
+    ) -> List[DocumentQuestionAnsweringOutputElement]:
         """
         Answer questions on document images.
 
@@ -506,7 +477,7 @@ class InferenceClient:
                 Defaults to None.
 
         Returns:
-            `List[Dict]`: a list of dictionaries containing the predicted label, associated probability, word ids, and page number.
+            `List[DocumentQuestionAnsweringOutputElement]`: a list of [`DocumentQuestionAnsweringOutputElement`] items containing the predicted label, associated probability, word ids, and page number.
 
         Raises:
             [`InferenceTimeoutError`]:
@@ -519,12 +490,12 @@ class InferenceClient:
         >>> from huggingface_hub import InferenceClient
         >>> client = InferenceClient()
         >>> client.document_question_answering(image="https://huggingface.co/spaces/impira/docquery/resolve/2359223c1837a7587402bda0f2643382a6eefeab/invoice.png", question="What is the invoice number?")
-        [{'score': 0.42515629529953003, 'answer': 'us-001', 'start': 16, 'end': 16}]
+        [DocumentQuestionAnsweringOutputElement(score=0.42515629529953003, answer='us-001', start=16, end=16)]
         ```
         """
         payload: Dict[str, Any] = {"question": question, "image": _b64_encode(image)}
         response = self.post(json=payload, model=model, task="document-question-answering")
-        return _bytes_to_list(response)
+        return DocumentQuestionAnsweringOutputElement.parse_obj_as_list(response)
 
     def feature_extraction(self, text: str, *, model: Optional[str] = None) -> "np.ndarray":
         """
@@ -562,7 +533,7 @@ class InferenceClient:
         np = _import_numpy()
         return np.array(_bytes_to_dict(response), dtype="float32")
 
-    def fill_mask(self, text: str, *, model: Optional[str] = None) -> List[FillMaskOutput]:
+    def fill_mask(self, text: str, *, model: Optional[str] = None) -> List[FillMaskOutputElement]:
         """
         Fill in a hole with a missing word (token to be precise).
 
@@ -575,7 +546,7 @@ class InferenceClient:
                 Defaults to None.
 
         Returns:
-            `List[Dict]`: a list of fill mask output dictionaries containing the predicted label, associated
+            `List[FillMaskOutputElement]`: a list of [`FillMaskOutputElement`] items containing the predicted label, associated
             probability, token reference, and completed text.
 
         Raises:
@@ -589,25 +560,21 @@ class InferenceClient:
         >>> from huggingface_hub import InferenceClient
         >>> client = InferenceClient()
         >>> client.fill_mask("The goal of life is <mask>.")
-        [{'score': 0.06897063553333282,
-        'token': 11098,
-        'token_str': ' happiness',
-        'sequence': 'The goal of life is happiness.'},
-        {'score': 0.06554922461509705,
-        'token': 45075,
-        'token_str': ' immortality',
-        'sequence': 'The goal of life is immortality.'}]
+        [
+            FillMaskOutputElement(score=0.06897063553333282, token=11098, token_str=' happiness', sequence='The goal of life is happiness.'),
+            FillMaskOutputElement(score=0.06554922461509705, token=45075, token_str=' immortality', sequence='The goal of life is immortality.')
+        ]
         ```
         """
         response = self.post(json={"inputs": text}, model=model, task="fill-mask")
-        return _bytes_to_list(response)
+        return FillMaskOutputElement.parse_obj_as_list(response)
 
     def image_classification(
         self,
         image: ContentT,
         *,
         model: Optional[str] = None,
-    ) -> List[ClassificationOutput]:
+    ) -> List[ImageClassificationOutputElement]:
         """
         Perform image classification on the given image using the specified model.
 
@@ -619,7 +586,7 @@ class InferenceClient:
                 deployed Inference Endpoint. If not provided, the default recommended model for image classification will be used.
 
         Returns:
-            `List[Dict]`: a list of dictionaries containing the predicted label and associated probability.
+            `List[ImageClassificationOutputElement]`: a list of [`ImageClassificationOutputElement`] items containing the predicted label and associated probability.
 
         Raises:
             [`InferenceTimeoutError`]:
@@ -632,18 +599,18 @@ class InferenceClient:
         >>> from huggingface_hub import InferenceClient
         >>> client = InferenceClient()
         >>> client.image_classification("https://upload.wikimedia.org/wikipedia/commons/thumb/4/43/Cute_dog.jpg/320px-Cute_dog.jpg")
-        [{'score': 0.9779096841812134, 'label': 'Blenheim spaniel'}, ...]
+        [ImageClassificationOutputElement(score=0.9779096841812134, label='Blenheim spaniel'), ...]
         ```
         """
         response = self.post(data=image, model=model, task="image-classification")
-        return _bytes_to_list(response)
+        return ImageClassificationOutputElement.parse_obj_as_list(response)
 
     def image_segmentation(
         self,
         image: ContentT,
         *,
         model: Optional[str] = None,
-    ) -> List[ImageSegmentationOutput]:
+    ) -> List[ImageSegmentationOutputElement]:
         """
         Perform image segmentation on the given image using the specified model.
 
@@ -661,7 +628,7 @@ class InferenceClient:
                 deployed Inference Endpoint. If not provided, the default recommended model for image segmentation will be used.
 
         Returns:
-            `List[Dict]`: A list of dictionaries containing the segmented masks and associated attributes.
+            `List[ImageSegmentationOutputElement]`: A list of [`ImageSegmentationOutputElement`] items containing the segmented masks and associated attributes.
 
         Raises:
             [`InferenceTimeoutError`]:
@@ -674,19 +641,13 @@ class InferenceClient:
         >>> from huggingface_hub import InferenceClient
         >>> client = InferenceClient()
         >>> client.image_segmentation("cat.jpg"):
-        [{'score': 0.989008, 'label': 'LABEL_184', 'mask': <PIL.PngImagePlugin.PngImageFile image mode=L size=400x300 at 0x7FDD2B129CC0>}, ...]
+        [ImageSegmentationOutputElement(score=0.989008, label='LABEL_184', mask=<PIL.PngImagePlugin.PngImageFile image mode=L size=400x300 at 0x7FDD2B129CC0>), ...]
         ```
         """
-
-        # Segment
         response = self.post(data=image, model=model, task="image-segmentation")
-        output = _bytes_to_dict(response)
-
-        # Parse masks as PIL Image
-        if not isinstance(output, list):
-            raise ValueError(f"Server output must be a list. Got {type(output)}: {str(output)[:200]}...")
+        output = ImageSegmentationOutputElement.parse_obj_as_list(response)
         for item in output:
-            item["mask"] = _b64_to_image(item["mask"])
+            item.mask = _b64_to_image(item.mask)
         return output
 
     def image_to_image(
@@ -773,7 +734,7 @@ class InferenceClient:
         response = self.post(json=payload, data=data, model=model, task="image-to-image")
         return _bytes_to_image(response)
 
-    def image_to_text(self, image: ContentT, *, model: Optional[str] = None) -> str:
+    def image_to_text(self, image: ContentT, *, model: Optional[str] = None) -> ImageToTextOutput:
         """
         Takes an input image and return text.
 
@@ -788,7 +749,7 @@ class InferenceClient:
                 Inference Endpoint. This parameter overrides the model defined at the instance level. Defaults to None.
 
         Returns:
-            `str`: The generated text.
+            [`ImageToTextOutput`]: The generated text.
 
         Raises:
             [`InferenceTimeoutError`]:
@@ -807,7 +768,7 @@ class InferenceClient:
         ```
         """
         response = self.post(data=image, model=model, task="image-to-text")
-        return _bytes_to_dict(response)[0]["generated_text"]
+        return ImageToTextOutput.parse_obj_as_instance(response)
 
     def list_deployed_models(
         self, frameworks: Union[None, str, Literal["all"], List[str]] = None
@@ -889,7 +850,7 @@ class InferenceClient:
         image: ContentT,
         *,
         model: Optional[str] = None,
-    ) -> List[ObjectDetectionOutput]:
+    ) -> List[ObjectDetectionOutputElement]:
         """
         Perform object detection on the given image using the specified model.
 
@@ -907,7 +868,7 @@ class InferenceClient:
                 deployed Inference Endpoint. If not provided, the default recommended model for object detection (DETR) will be used.
 
         Returns:
-            `List[ObjectDetectionOutput]`: A list of dictionaries containing the bounding boxes and associated attributes.
+            `List[ObjectDetectionOutputElement]`: A list of [`ObjectDetectionOutputElement`] items containing the bounding boxes and associated attributes.
 
         Raises:
             [`InferenceTimeoutError`]:
@@ -922,19 +883,16 @@ class InferenceClient:
         >>> from huggingface_hub import InferenceClient
         >>> client = InferenceClient()
         >>> client.object_detection("people.jpg"):
-        [{"score":0.9486683011054993,"label":"person","box":{"xmin":59,"ymin":39,"xmax":420,"ymax":510}}, ... ]
+        [ObjectDetectionOutputElement(score=0.9486683011054993, label='person', box=BoundingBox(xmin=59, ymin=39, xmax=420, ymax=510)), ...]
         ```
         """
         # detect objects
         response = self.post(data=image, model=model, task="object-detection")
-        output = _bytes_to_dict(response)
-        if not isinstance(output, list):
-            raise ValueError(f"Server output must be a list. Got {type(output)}: {str(output)[:200]}...")
-        return output
+        return ObjectDetectionOutputElement.parse_obj_as_list(response)
 
     def question_answering(
         self, question: str, context: str, *, model: Optional[str] = None
-    ) -> QuestionAnsweringOutput:
+    ) -> QuestionAnsweringOutputElement:
         """
         Retrieve the answer to a question from a given text.
 
@@ -948,7 +906,7 @@ class InferenceClient:
                 a deployed Inference Endpoint.
 
         Returns:
-            `Dict`: a dictionary of question answering output containing the score, start index, end index, and answer.
+            [`QuestionAnsweringOutputElement`]: an question answering output containing the score, start index, end index, and answer.
 
         Raises:
             [`InferenceTimeoutError`]:
@@ -961,7 +919,7 @@ class InferenceClient:
         >>> from huggingface_hub import InferenceClient
         >>> client = InferenceClient()
         >>> client.question_answering(question="What's my name?", context="My name is Clara and I live in Berkeley.")
-        {'score': 0.9326562285423279, 'start': 11, 'end': 16, 'answer': 'Clara'}
+        QuestionAnsweringOutputElement(score=0.9326562285423279, start=11, end=16, answer='Clara')
         ```
         """
 
@@ -971,7 +929,7 @@ class InferenceClient:
             model=model,
             task="question-answering",
         )
-        return _bytes_to_dict(response)  # type: ignore
+        return QuestionAnsweringOutputElement.parse_obj_as_instance(response)
 
     def sentence_similarity(
         self, sentence: str, other_sentences: List[str], *, model: Optional[str] = None
@@ -1026,7 +984,7 @@ class InferenceClient:
         *,
         parameters: Optional[Dict[str, Any]] = None,
         model: Optional[str] = None,
-    ) -> str:
+    ) -> SummarizationOutput:
         """
         Generate a summary of a given text using a specified model.
 
@@ -1041,7 +999,7 @@ class InferenceClient:
                 Inference Endpoint. This parameter overrides the model defined at the instance level. Defaults to None.
 
         Returns:
-            `str`: The generated summary text.
+            [`SummarizationOutput`]: The generated summary text.
 
         Raises:
             [`InferenceTimeoutError`]:
@@ -1054,18 +1012,18 @@ class InferenceClient:
         >>> from huggingface_hub import InferenceClient
         >>> client = InferenceClient()
         >>> client.summarization("The Eiffel tower...")
-        'The Eiffel tower is one of the most famous landmarks in the world....'
+        SummarizationOutput(generated_text="The Eiffel tower is one of the most famous landmarks in the world....")
         ```
         """
         payload: Dict[str, Any] = {"inputs": text}
         if parameters is not None:
             payload["parameters"] = parameters
         response = self.post(json=payload, model=model, task="summarization")
-        return _bytes_to_dict(response)[0]["summary_text"]
+        return SummarizationOutput.parse_obj_as_list(response)[0]
 
     def table_question_answering(
         self, table: Dict[str, Any], query: str, *, model: Optional[str] = None
-    ) -> TableQuestionAnsweringOutput:
+    ) -> TableQuestionAnsweringOutputElement:
         """
         Retrieve the answer to a question from information given in a table.
 
@@ -1080,7 +1038,7 @@ class InferenceClient:
                 Hub or a URL to a deployed Inference Endpoint.
 
         Returns:
-            `Dict`: a dictionary of table question answering output containing the answer, coordinates, cells and the aggregator used.
+            [`TableQuestionAnsweringOutputElement`]: a table question answering output containing the answer, coordinates, cells and the aggregator used.
 
         Raises:
             [`InferenceTimeoutError`]:
@@ -1095,7 +1053,7 @@ class InferenceClient:
         >>> query = "How many stars does the transformers repository have?"
         >>> table = {"Repository": ["Transformers", "Datasets", "Tokenizers"], "Stars": ["36542", "4512", "3934"]}
         >>> client.table_question_answering(table, query, model="google/tapas-base-finetuned-wtq")
-        {'answer': 'AVERAGE > 36542', 'coordinates': [[0, 1]], 'cells': ['36542'], 'aggregator': 'AVERAGE'}
+        TableQuestionAnsweringOutputElement(answer='36542', coordinates=[[0, 1]], cells=['36542'], aggregator='AVERAGE')
         ```
         """
         response = self.post(
@@ -1106,7 +1064,7 @@ class InferenceClient:
             model=model,
             task="table-question-answering",
         )
-        return _bytes_to_dict(response)  # type: ignore
+        return TableQuestionAnsweringOutputElement.parse_obj_as_instance(response)
 
     def tabular_classification(self, table: Dict[str, Any], *, model: Optional[str] = None) -> List[str]:
         """
@@ -1193,7 +1151,7 @@ class InferenceClient:
         response = self.post(json={"table": table}, model=model, task="tabular-regression")
         return _bytes_to_list(response)
 
-    def text_classification(self, text: str, *, model: Optional[str] = None) -> List[ClassificationOutput]:
+    def text_classification(self, text: str, *, model: Optional[str] = None) -> List[TextClassificationOutputElement]:
         """
         Perform text classification (e.g. sentiment-analysis) on the given text.
 
@@ -1206,7 +1164,7 @@ class InferenceClient:
                 Defaults to None.
 
         Returns:
-            `List[Dict]`: a list of dictionaries containing the predicted label and associated probability.
+            `List[TextClassificationOutputElement]`: a list of [`TextClassificationOutputElement`] items containing the predicted label and associated probability.
 
         Raises:
             [`InferenceTimeoutError`]:
@@ -1219,11 +1177,14 @@ class InferenceClient:
         >>> from huggingface_hub import InferenceClient
         >>> client = InferenceClient()
         >>> client.text_classification("I like you")
-        [{'label': 'POSITIVE', 'score': 0.9998695850372314}, {'label': 'NEGATIVE', 'score': 0.0001304351753788069}]
+        [
+            TextClassificationOutputElement(label='POSITIVE', score=0.9998695850372314),
+            TextClassificationOutputElement(label='NEGATIVE', score=0.0001304351753788069),
+        ]
         ```
         """
         response = self.post(json={"inputs": text}, model=model, task="text-classification")
-        return _bytes_to_list(response)[0]
+        return TextClassificationOutputElement.parse_obj_as_list(response)[0]  # type: ignore [return-value]
 
     @overload
     def text_generation(  # type: ignore
@@ -1700,7 +1661,9 @@ class InferenceClient:
         """
         return self.post(json={"inputs": text}, model=model, task="text-to-speech")
 
-    def token_classification(self, text: str, *, model: Optional[str] = None) -> List[TokenClassificationOutput]:
+    def token_classification(
+        self, text: str, *, model: Optional[str] = None
+    ) -> List[TokenClassificationOutputElement]:
         """
         Perform token classification on the given text.
         Usually used for sentence parsing, either grammatical, or Named Entity Recognition (NER) to understand keywords contained within text.
@@ -1714,7 +1677,7 @@ class InferenceClient:
                 Defaults to None.
 
         Returns:
-            `List[Dict]`: List of token classification outputs containing the entity group, confidence score, word, start and end index.
+            `List[TokenClassificationOutputElement]`: List of [`TokenClassificationOutputElement`] items containing the entity group, confidence score, word, start and end index.
 
         Raises:
             [`InferenceTimeoutError`]:
@@ -1727,16 +1690,22 @@ class InferenceClient:
         >>> from huggingface_hub import InferenceClient
         >>> client = InferenceClient()
         >>> client.token_classification("My name is Sarah Jessica Parker but you can call me Jessica")
-        [{'entity_group': 'PER',
-        'score': 0.9971321225166321,
-        'word': 'Sarah Jessica Parker',
-        'start': 11,
-        'end': 31},
-        {'entity_group': 'PER',
-        'score': 0.9773476123809814,
-        'word': 'Jessica',
-        'start': 52,
-        'end': 59}]
+        [
+            TokenClassificationOutputElement(
+                entity_group='PER',
+                score=0.9971321225166321,
+                word='Sarah Jessica Parker',
+                start=11,
+                end=31,
+            ),
+            TokenClassificationOutputElement(
+                entity_group='PER',
+                score=0.9773476123809814,
+                word='Jessica',
+                start=52,
+                end=59,
+            )
+        ]
         ```
         """
         payload: Dict[str, Any] = {"inputs": text}
@@ -1745,11 +1714,11 @@ class InferenceClient:
             model=model,
             task="token-classification",
         )
-        return _bytes_to_list(response)
+        return TokenClassificationOutputElement.parse_obj_as_list(response)
 
     def translation(
         self, text: str, *, model: Optional[str] = None, src_lang: Optional[str] = None, tgt_lang: Optional[str] = None
-    ) -> str:
+    ) -> TranslationOutput:
         """
         Convert text from one language to another.
 
@@ -1772,7 +1741,7 @@ class InferenceClient:
                 Target language of the translation task, i.e. output language. Cannot be passed without `src_lang`.
 
         Returns:
-            `str`: The generated translated text.
+            [`TranslationOutput`]: The generated translated text.
 
         Raises:
             [`InferenceTimeoutError`]:
@@ -1789,7 +1758,7 @@ class InferenceClient:
         >>> client.translation("My name is Wolfgang and I live in Berlin")
         'Mein Name ist Wolfgang und ich lebe in Berlin.'
         >>> client.translation("My name is Wolfgang and I live in Berlin", model="Helsinki-NLP/opus-mt-en-fr")
-        "Je m'appelle Wolfgang et je vis à Berlin."
+        TranslationOutput(translation_text='Je m\'appelle Wolfgang et je vis à Berlin.')
         ```
 
         Specifying languages:
@@ -1810,11 +1779,58 @@ class InferenceClient:
         if src_lang and tgt_lang:
             payload["parameters"] = {"src_lang": src_lang, "tgt_lang": tgt_lang}
         response = self.post(json=payload, model=model, task="translation")
-        return _bytes_to_dict(response)[0]["translation_text"]
+        return TranslationOutput.parse_obj_as_list(response)[0]
+
+    def visual_question_answering(
+        self,
+        image: ContentT,
+        question: str,
+        *,
+        model: Optional[str] = None,
+    ) -> List[VisualQuestionAnsweringOutputElement]:
+        """
+        Answering open-ended questions based on an image.
+
+        Args:
+            image (`Union[str, Path, bytes, BinaryIO]`):
+                The input image for the context. It can be raw bytes, an image file, or a URL to an online image.
+            question (`str`):
+                Question to be answered.
+            model (`str`, *optional*):
+                The model to use for the visual question answering task. Can be a model ID hosted on the Hugging Face Hub or a URL to
+                a deployed Inference Endpoint. If not provided, the default recommended visual question answering model will be used.
+                Defaults to None.
+
+        Returns:
+            `List[VisualQuestionAnsweringOutputElement]`: a list of [`VisualQuestionAnsweringOutputElement`] items containing the predicted label and associated probability.
+
+        Raises:
+            `InferenceTimeoutError`:
+                If the model is unavailable or the request times out.
+            `HTTPError`:
+                If the request fails with an HTTP error status code other than HTTP 503.
+
+        Example:
+        ```py
+        >>> from huggingface_hub import InferenceClient
+        >>> client = InferenceClient()
+        >>> client.visual_question_answering(
+        ...     image="https://huggingface.co/datasets/mishig/sample_images/resolve/main/tiger.jpg",
+        ...     question="What is the animal doing?"
+        ... )
+        [
+            VisualQuestionAnsweringOutputElement(score=0.778609573841095, answer='laying down'),
+            VisualQuestionAnsweringOutputElement(score=0.6957435607910156, answer='sitting'),
+        ]
+        ```
+        """
+        payload: Dict[str, Any] = {"question": question, "image": _b64_encode(image)}
+        response = self.post(json=payload, model=model, task="visual-question-answering")
+        return VisualQuestionAnsweringOutputElement.parse_obj_as_list(response)
 
     def zero_shot_classification(
         self, text: str, labels: List[str], *, multi_label: bool = False, model: Optional[str] = None
-    ) -> List[ClassificationOutput]:
+    ) -> List[ZeroShotClassificationOutputElement]:
         """
         Provide as input a text and a set of candidate labels to classify the input text.
 
@@ -1830,7 +1846,7 @@ class InferenceClient:
                 Inference Endpoint. This parameter overrides the model defined at the instance level. Defaults to None.
 
         Returns:
-            `List[Dict]`: List of classification outputs containing the predicted labels and their confidence.
+            `List[ZeroShotClassificationOutputElement]`: List of [`ZeroShotClassificationOutputElement`] items containing the predicted labels and their confidence.
 
         Raises:
             [`InferenceTimeoutError`]:
@@ -1850,19 +1866,19 @@ class InferenceClient:
         >>> labels = ["space & cosmos", "scientific discovery", "microbiology", "robots", "archeology"]
         >>> client.zero_shot_classification(text, labels)
         [
-            {"label": "scientific discovery", "score": 0.7961668968200684},
-            {"label": "space & cosmos", "score": 0.18570658564567566},
-            {"label": "microbiology", "score": 0.00730885099619627},
-            {"label": "archeology", "score": 0.006258360575884581},
-            {"label": "robots", "score": 0.004559356719255447},
+            ZeroShotClassificationOutputElement(label='scientific discovery', score=0.7961668968200684),
+            ZeroShotClassificationOutputElement(label='space & cosmos', score=0.18570658564567566),
+            ZeroShotClassificationOutputElement(label='microbiology', score=0.00730885099619627),
+            ZeroShotClassificationOutputElement(label='archeology', score=0.006258360575884581),
+            ZeroShotClassificationOutputElement(label='robots', score=0.004559356719255447),
         ]
         >>> client.zero_shot_classification(text, labels, multi_label=True)
         [
-            {"label": "scientific discovery", "score": 0.9829297661781311},
-            {"label": "space & cosmos", "score": 0.755190908908844},
-            {"label": "microbiology", "score": 0.0005462635890580714},
-            {"label": "archeology", "score": 0.00047131875180639327},
-            {"label": "robots", "score": 0.00030448526376858354},
+            ZeroShotClassificationOutputElement(label='scientific discovery', score=0.9829297661781311),
+            ZeroShotClassificationOutputElement(label='space & cosmos', score=0.755190908908844),
+            ZeroShotClassificationOutputElement(label='microbiology', score=0.0005462635890580714),
+            ZeroShotClassificationOutputElement(label='archeology', score=0.00047131875180639327),
+            ZeroShotClassificationOutputElement(label='robots', score=0.00030448526376858354),
         ]
         ```
         """
@@ -1882,11 +1898,14 @@ class InferenceClient:
             task="zero-shot-classification",
         )
         output = _bytes_to_dict(response)
-        return [{"label": label, "score": score} for label, score in zip(output["labels"], output["scores"])]
+        return [
+            ZeroShotClassificationOutputElement.parse_obj_as_instance({"label": label, "score": score})
+            for label, score in zip(output["labels"], output["scores"])
+        ]
 
     def zero_shot_image_classification(
         self, image: ContentT, labels: List[str], *, model: Optional[str] = None
-    ) -> List[ClassificationOutput]:
+    ) -> List[ZeroShotImageClassificationOutputElement]:
         """
         Provide input image and text labels to predict text labels for the image.
 
@@ -1900,7 +1919,7 @@ class InferenceClient:
                 Inference Endpoint. This parameter overrides the model defined at the instance level. Defaults to None.
 
         Returns:
-            `List[Dict]`: List of classification outputs containing the predicted labels and their confidence.
+            `List[ZeroShotImageClassificationOutputElement]`: List of [`ZeroShotImageClassificationOutputElement`] items containing the predicted labels and their confidence.
 
         Raises:
             [`InferenceTimeoutError`]:
@@ -1917,7 +1936,7 @@ class InferenceClient:
         ...     "https://upload.wikimedia.org/wikipedia/commons/thumb/4/43/Cute_dog.jpg/320px-Cute_dog.jpg",
         ...     labels=["dog", "cat", "horse"],
         ... )
-        [{"label": "dog", "score": 0.956}, ...]
+        [ZeroShotImageClassificationOutputElement(label='dog', score=0.956),...]
         ```
         """
         # Raise ValueError if input is less than 2 labels
@@ -1929,7 +1948,7 @@ class InferenceClient:
             model=model,
             task="zero-shot-image-classification",
         )
-        return _bytes_to_list(response)
+        return ZeroShotImageClassificationOutputElement.parse_obj_as_list(response)
 
     def _resolve_url(self, model: Optional[str] = None, task: Optional[str] = None) -> str:
         model = model or self.model
diff --git a/src/huggingface_hub/inference/_generated/_async_client.py b/src/huggingface_hub/inference/_generated/_async_client.py
index b5e83cdd..cb3f8fdb 100644
--- a/src/huggingface_hub/inference/_generated/_async_client.py
+++ b/src/huggingface_hub/inference/_generated/_async_client.py
@@ -56,6 +56,26 @@ from huggingface_hub.inference._common import (
     _open_as_binary,
     _set_as_non_tgi,
 )
+from huggingface_hub.inference._generated.types import (
+    AudioClassificationOutputElement,
+    AudioToAudioOutputElement,
+    AutomaticSpeechRecognitionOutput,
+    DocumentQuestionAnsweringOutputElement,
+    FillMaskOutputElement,
+    ImageClassificationOutputElement,
+    ImageSegmentationOutputElement,
+    ImageToTextOutput,
+    ObjectDetectionOutputElement,
+    QuestionAnsweringOutputElement,
+    SummarizationOutput,
+    TableQuestionAnsweringOutputElement,
+    TextClassificationOutputElement,
+    TokenClassificationOutputElement,
+    TranslationOutput,
+    VisualQuestionAnsweringOutputElement,
+    ZeroShotClassificationOutputElement,
+    ZeroShotImageClassificationOutputElement,
+)
 from huggingface_hub.inference._text_generation import (
     TextGenerationParameters,
     TextGenerationRequest,
@@ -64,15 +84,7 @@ from huggingface_hub.inference._text_generation import (
     raise_text_generation_error,
 )
 from huggingface_hub.inference._types import (
-    AudioToAudioOutput,
-    ClassificationOutput,
-    ConversationalOutput,
-    FillMaskOutput,
-    ImageSegmentationOutput,
-    ObjectDetectionOutput,
-    QuestionAnsweringOutput,
-    TableQuestionAnsweringOutput,
-    TokenClassificationOutput,
+    ConversationalOutput,  # soon to be removed
 )
 from huggingface_hub.utils import (
     build_hf_headers,
@@ -263,7 +275,7 @@ class AsyncInferenceClient:
         audio: ContentT,
         *,
         model: Optional[str] = None,
-    ) -> List[ClassificationOutput]:
+    ) -> List[AudioClassificationOutputElement]:
         """
         Perform audio classification on the provided audio content.
 
@@ -277,7 +289,7 @@ class AsyncInferenceClient:
                 audio classification will be used.
 
         Returns:
-            `List[Dict]`: The classification output containing the predicted label and its confidence.
+            `List[AudioClassificationOutputElement]`: List of [`AudioClassificationOutputElement`] items containing the predicted labels and their confidence.
 
         Raises:
             [`InferenceTimeoutError`]:
@@ -291,18 +303,22 @@ class AsyncInferenceClient:
         >>> from huggingface_hub import AsyncInferenceClient
         >>> client = AsyncInferenceClient()
         >>> await client.audio_classification("audio.flac")
-        [{'score': 0.4976358711719513, 'label': 'hap'}, {'score': 0.3677836060523987, 'label': 'neu'},...]
+        [
+            AudioClassificationOutputElement(score=0.4976358711719513, label='hap'),
+            AudioClassificationOutputElement(score=0.3677836060523987, label='neu'),
+            ...
+        ]
         ```
         """
         response = await self.post(data=audio, model=model, task="audio-classification")
-        return _bytes_to_list(response)
+        return AudioClassificationOutputElement.parse_obj_as_list(response)
 
     async def audio_to_audio(
         self,
         audio: ContentT,
         *,
         model: Optional[str] = None,
-    ) -> List[AudioToAudioOutput]:
+    ) -> List[AudioToAudioOutputElement]:
         """
         Performs multiple tasks related to audio-to-audio depending on the model (eg: speech enhancement, source separation).
 
@@ -316,7 +332,7 @@ class AsyncInferenceClient:
                 audio_to_audio will be used.
 
         Returns:
-            `List[Dict]`: A list of dictionary where each index contains audios label, content-type, and audio content in blob.
+            `List[AudioToAudioOutputElement]`: A list of [`AudioToAudioOutputElement`] items containing audios label, content-type, and audio content in blob.
 
         Raises:
             `InferenceTimeoutError`:
@@ -332,13 +348,13 @@ class AsyncInferenceClient:
         >>> audio_output = await client.audio_to_audio("audio.flac")
         >>> async for i, item in enumerate(audio_output):
         >>>     with open(f"output_{i}.flac", "wb") as f:
-                    f.write(item["blob"])
+                    f.write(item.blob)
         ```
         """
         response = await self.post(data=audio, model=model, task="audio-to-audio")
-        audio_output = _bytes_to_list(response)
+        audio_output = AudioToAudioOutputElement.parse_obj_as_list(response)
         for item in audio_output:
-            item["blob"] = base64.b64decode(item["blob"])
+            item.blob = base64.b64decode(item.blob)
         return audio_output
 
     async def automatic_speech_recognition(
@@ -346,7 +362,7 @@ class AsyncInferenceClient:
         audio: ContentT,
         *,
         model: Optional[str] = None,
-    ) -> str:
+    ) -> AutomaticSpeechRecognitionOutput:
         """
         Perform automatic speech recognition (ASR or audio-to-text) on the given audio content.
 
@@ -358,7 +374,7 @@ class AsyncInferenceClient:
                 Inference Endpoint. If not provided, the default recommended model for ASR will be used.
 
         Returns:
-            str: The transcribed text.
+            [`AutomaticSpeechRecognitionOutput`]: An item containing the transcribed text and optionally the timestamp chunks.
 
         Raises:
             [`InferenceTimeoutError`]:
@@ -371,12 +387,12 @@ class AsyncInferenceClient:
         # Must be run in an async context
         >>> from huggingface_hub import AsyncInferenceClient
         >>> client = AsyncInferenceClient()
-        >>> await client.automatic_speech_recognition("hello_world.flac")
+        >>> await client.automatic_speech_recognition("hello_world.flac").text
         "hello world"
         ```
         """
         response = await self.post(data=audio, model=model, task="automatic-speech-recognition")
-        return _bytes_to_dict(response)["text"]
+        return AutomaticSpeechRecognitionOutput.parse_obj_as_instance(response)
 
     async def conversational(
         self,
@@ -440,58 +456,13 @@ class AsyncInferenceClient:
         response = await self.post(json=payload, model=model, task="conversational")
         return _bytes_to_dict(response)  # type: ignore
 
-    async def visual_question_answering(
-        self,
-        image: ContentT,
-        question: str,
-        *,
-        model: Optional[str] = None,
-    ) -> List[str]:
-        """
-        Answering open-ended questions based on an image.
-
-        Args:
-            image (`Union[str, Path, bytes, BinaryIO]`):
-                The input image for the context. It can be raw bytes, an image file, or a URL to an online image.
-            question (`str`):
-                Question to be answered.
-            model (`str`, *optional*):
-                The model to use for the visual question answering task. Can be a model ID hosted on the Hugging Face Hub or a URL to
-                a deployed Inference Endpoint. If not provided, the default recommended visual question answering model will be used.
-                Defaults to None.
-
-        Returns:
-            `List[Dict]`: a list of dictionaries containing the predicted label and associated probability.
-
-        Raises:
-            `InferenceTimeoutError`:
-                If the model is unavailable or the request times out.
-            `aiohttp.ClientResponseError`:
-                If the request fails with an HTTP error status code other than HTTP 503.
-
-        Example:
-        ```py
-        # Must be run in an async context
-        >>> from huggingface_hub import AsyncInferenceClient
-        >>> client = AsyncInferenceClient()
-        >>> await client.visual_question_answering(
-        ...     image="https://huggingface.co/datasets/mishig/sample_images/resolve/main/tiger.jpg",
-        ...     question="What is the animal doing?"
-        ... )
-        [{'score': 0.778609573841095, 'answer': 'laying down'},{'score': 0.6957435607910156, 'answer': 'sitting'}, ...]
-        ```
-        """
-        payload: Dict[str, Any] = {"question": question, "image": _b64_encode(image)}
-        response = await self.post(json=payload, model=model, task="visual-question-answering")
-        return _bytes_to_list(response)
-
     async def document_question_answering(
         self,
         image: ContentT,
         question: str,
         *,
         model: Optional[str] = None,
-    ) -> List[QuestionAnsweringOutput]:
+    ) -> List[DocumentQuestionAnsweringOutputElement]:
         """
         Answer questions on document images.
 
@@ -506,7 +477,7 @@ class AsyncInferenceClient:
                 Defaults to None.
 
         Returns:
-            `List[Dict]`: a list of dictionaries containing the predicted label, associated probability, word ids, and page number.
+            `List[DocumentQuestionAnsweringOutputElement]`: a list of [`DocumentQuestionAnsweringOutputElement`] items containing the predicted label, associated probability, word ids, and page number.
 
         Raises:
             [`InferenceTimeoutError`]:
@@ -520,12 +491,12 @@ class AsyncInferenceClient:
         >>> from huggingface_hub import AsyncInferenceClient
         >>> client = AsyncInferenceClient()
         >>> await client.document_question_answering(image="https://huggingface.co/spaces/impira/docquery/resolve/2359223c1837a7587402bda0f2643382a6eefeab/invoice.png", question="What is the invoice number?")
-        [{'score': 0.42515629529953003, 'answer': 'us-001', 'start': 16, 'end': 16}]
+        [DocumentQuestionAnsweringOutputElement(score=0.42515629529953003, answer='us-001', start=16, end=16)]
         ```
         """
         payload: Dict[str, Any] = {"question": question, "image": _b64_encode(image)}
         response = await self.post(json=payload, model=model, task="document-question-answering")
-        return _bytes_to_list(response)
+        return DocumentQuestionAnsweringOutputElement.parse_obj_as_list(response)
 
     async def feature_extraction(self, text: str, *, model: Optional[str] = None) -> "np.ndarray":
         """
@@ -564,7 +535,7 @@ class AsyncInferenceClient:
         np = _import_numpy()
         return np.array(_bytes_to_dict(response), dtype="float32")
 
-    async def fill_mask(self, text: str, *, model: Optional[str] = None) -> List[FillMaskOutput]:
+    async def fill_mask(self, text: str, *, model: Optional[str] = None) -> List[FillMaskOutputElement]:
         """
         Fill in a hole with a missing word (token to be precise).
 
@@ -577,7 +548,7 @@ class AsyncInferenceClient:
                 Defaults to None.
 
         Returns:
-            `List[Dict]`: a list of fill mask output dictionaries containing the predicted label, associated
+            `List[FillMaskOutputElement]`: a list of [`FillMaskOutputElement`] items containing the predicted label, associated
             probability, token reference, and completed text.
 
         Raises:
@@ -592,25 +563,21 @@ class AsyncInferenceClient:
         >>> from huggingface_hub import AsyncInferenceClient
         >>> client = AsyncInferenceClient()
         >>> await client.fill_mask("The goal of life is <mask>.")
-        [{'score': 0.06897063553333282,
-        'token': 11098,
-        'token_str': ' happiness',
-        'sequence': 'The goal of life is happiness.'},
-        {'score': 0.06554922461509705,
-        'token': 45075,
-        'token_str': ' immortality',
-        'sequence': 'The goal of life is immortality.'}]
+        [
+            FillMaskOutputElement(score=0.06897063553333282, token=11098, token_str=' happiness', sequence='The goal of life is happiness.'),
+            FillMaskOutputElement(score=0.06554922461509705, token=45075, token_str=' immortality', sequence='The goal of life is immortality.')
+        ]
         ```
         """
         response = await self.post(json={"inputs": text}, model=model, task="fill-mask")
-        return _bytes_to_list(response)
+        return FillMaskOutputElement.parse_obj_as_list(response)
 
     async def image_classification(
         self,
         image: ContentT,
         *,
         model: Optional[str] = None,
-    ) -> List[ClassificationOutput]:
+    ) -> List[ImageClassificationOutputElement]:
         """
         Perform image classification on the given image using the specified model.
 
@@ -622,7 +589,7 @@ class AsyncInferenceClient:
                 deployed Inference Endpoint. If not provided, the default recommended model for image classification will be used.
 
         Returns:
-            `List[Dict]`: a list of dictionaries containing the predicted label and associated probability.
+            `List[ImageClassificationOutputElement]`: a list of [`ImageClassificationOutputElement`] items containing the predicted label and associated probability.
 
         Raises:
             [`InferenceTimeoutError`]:
@@ -636,18 +603,18 @@ class AsyncInferenceClient:
         >>> from huggingface_hub import AsyncInferenceClient
         >>> client = AsyncInferenceClient()
         >>> await client.image_classification("https://upload.wikimedia.org/wikipedia/commons/thumb/4/43/Cute_dog.jpg/320px-Cute_dog.jpg")
-        [{'score': 0.9779096841812134, 'label': 'Blenheim spaniel'}, ...]
+        [ImageClassificationOutputElement(score=0.9779096841812134, label='Blenheim spaniel'), ...]
         ```
         """
         response = await self.post(data=image, model=model, task="image-classification")
-        return _bytes_to_list(response)
+        return ImageClassificationOutputElement.parse_obj_as_list(response)
 
     async def image_segmentation(
         self,
         image: ContentT,
         *,
         model: Optional[str] = None,
-    ) -> List[ImageSegmentationOutput]:
+    ) -> List[ImageSegmentationOutputElement]:
         """
         Perform image segmentation on the given image using the specified model.
 
@@ -665,7 +632,7 @@ class AsyncInferenceClient:
                 deployed Inference Endpoint. If not provided, the default recommended model for image segmentation will be used.
 
         Returns:
-            `List[Dict]`: A list of dictionaries containing the segmented masks and associated attributes.
+            `List[ImageSegmentationOutputElement]`: A list of [`ImageSegmentationOutputElement`] items containing the segmented masks and associated attributes.
 
         Raises:
             [`InferenceTimeoutError`]:
@@ -679,19 +646,13 @@ class AsyncInferenceClient:
         >>> from huggingface_hub import AsyncInferenceClient
         >>> client = AsyncInferenceClient()
         >>> await client.image_segmentation("cat.jpg"):
-        [{'score': 0.989008, 'label': 'LABEL_184', 'mask': <PIL.PngImagePlugin.PngImageFile image mode=L size=400x300 at 0x7FDD2B129CC0>}, ...]
+        [ImageSegmentationOutputElement(score=0.989008, label='LABEL_184', mask=<PIL.PngImagePlugin.PngImageFile image mode=L size=400x300 at 0x7FDD2B129CC0>), ...]
         ```
         """
-
-        # Segment
         response = await self.post(data=image, model=model, task="image-segmentation")
-        output = _bytes_to_dict(response)
-
-        # Parse masks as PIL Image
-        if not isinstance(output, list):
-            raise ValueError(f"Server output must be a list. Got {type(output)}: {str(output)[:200]}...")
+        output = ImageSegmentationOutputElement.parse_obj_as_list(response)
         for item in output:
-            item["mask"] = _b64_to_image(item["mask"])
+            item.mask = _b64_to_image(item.mask)
         return output
 
     async def image_to_image(
@@ -779,7 +740,7 @@ class AsyncInferenceClient:
         response = await self.post(json=payload, data=data, model=model, task="image-to-image")
         return _bytes_to_image(response)
 
-    async def image_to_text(self, image: ContentT, *, model: Optional[str] = None) -> str:
+    async def image_to_text(self, image: ContentT, *, model: Optional[str] = None) -> ImageToTextOutput:
         """
         Takes an input image and return text.
 
@@ -794,7 +755,7 @@ class AsyncInferenceClient:
                 Inference Endpoint. This parameter overrides the model defined at the instance level. Defaults to None.
 
         Returns:
-            `str`: The generated text.
+            [`ImageToTextOutput`]: The generated text.
 
         Raises:
             [`InferenceTimeoutError`]:
@@ -814,7 +775,7 @@ class AsyncInferenceClient:
         ```
         """
         response = await self.post(data=image, model=model, task="image-to-text")
-        return _bytes_to_dict(response)[0]["generated_text"]
+        return ImageToTextOutput.parse_obj_as_instance(response)
 
     async def list_deployed_models(
         self, frameworks: Union[None, str, Literal["all"], List[str]] = None
@@ -902,7 +863,7 @@ class AsyncInferenceClient:
         image: ContentT,
         *,
         model: Optional[str] = None,
-    ) -> List[ObjectDetectionOutput]:
+    ) -> List[ObjectDetectionOutputElement]:
         """
         Perform object detection on the given image using the specified model.
 
@@ -920,7 +881,7 @@ class AsyncInferenceClient:
                 deployed Inference Endpoint. If not provided, the default recommended model for object detection (DETR) will be used.
 
         Returns:
-            `List[ObjectDetectionOutput]`: A list of dictionaries containing the bounding boxes and associated attributes.
+            `List[ObjectDetectionOutputElement]`: A list of [`ObjectDetectionOutputElement`] items containing the bounding boxes and associated attributes.
 
         Raises:
             [`InferenceTimeoutError`]:
@@ -936,19 +897,16 @@ class AsyncInferenceClient:
         >>> from huggingface_hub import AsyncInferenceClient
         >>> client = AsyncInferenceClient()
         >>> await client.object_detection("people.jpg"):
-        [{"score":0.9486683011054993,"label":"person","box":{"xmin":59,"ymin":39,"xmax":420,"ymax":510}}, ... ]
+        [ObjectDetectionOutputElement(score=0.9486683011054993, label='person', box=BoundingBox(xmin=59, ymin=39, xmax=420, ymax=510)), ...]
         ```
         """
         # detect objects
         response = await self.post(data=image, model=model, task="object-detection")
-        output = _bytes_to_dict(response)
-        if not isinstance(output, list):
-            raise ValueError(f"Server output must be a list. Got {type(output)}: {str(output)[:200]}...")
-        return output
+        return ObjectDetectionOutputElement.parse_obj_as_list(response)
 
     async def question_answering(
         self, question: str, context: str, *, model: Optional[str] = None
-    ) -> QuestionAnsweringOutput:
+    ) -> QuestionAnsweringOutputElement:
         """
         Retrieve the answer to a question from a given text.
 
@@ -962,7 +920,7 @@ class AsyncInferenceClient:
                 a deployed Inference Endpoint.
 
         Returns:
-            `Dict`: a dictionary of question answering output containing the score, start index, end index, and answer.
+            [`QuestionAnsweringOutputElement`]: an question answering output containing the score, start index, end index, and answer.
 
         Raises:
             [`InferenceTimeoutError`]:
@@ -976,7 +934,7 @@ class AsyncInferenceClient:
         >>> from huggingface_hub import AsyncInferenceClient
         >>> client = AsyncInferenceClient()
         >>> await client.question_answering(question="What's my name?", context="My name is Clara and I live in Berkeley.")
-        {'score': 0.9326562285423279, 'start': 11, 'end': 16, 'answer': 'Clara'}
+        QuestionAnsweringOutputElement(score=0.9326562285423279, start=11, end=16, answer='Clara')
         ```
         """
 
@@ -986,7 +944,7 @@ class AsyncInferenceClient:
             model=model,
             task="question-answering",
         )
-        return _bytes_to_dict(response)  # type: ignore
+        return QuestionAnsweringOutputElement.parse_obj_as_instance(response)
 
     async def sentence_similarity(
         self, sentence: str, other_sentences: List[str], *, model: Optional[str] = None
@@ -1042,7 +1000,7 @@ class AsyncInferenceClient:
         *,
         parameters: Optional[Dict[str, Any]] = None,
         model: Optional[str] = None,
-    ) -> str:
+    ) -> SummarizationOutput:
         """
         Generate a summary of a given text using a specified model.
 
@@ -1057,7 +1015,7 @@ class AsyncInferenceClient:
                 Inference Endpoint. This parameter overrides the model defined at the instance level. Defaults to None.
 
         Returns:
-            `str`: The generated summary text.
+            [`SummarizationOutput`]: The generated summary text.
 
         Raises:
             [`InferenceTimeoutError`]:
@@ -1071,18 +1029,18 @@ class AsyncInferenceClient:
         >>> from huggingface_hub import AsyncInferenceClient
         >>> client = AsyncInferenceClient()
         >>> await client.summarization("The Eiffel tower...")
-        'The Eiffel tower is one of the most famous landmarks in the world....'
+        SummarizationOutput(generated_text="The Eiffel tower is one of the most famous landmarks in the world....")
         ```
         """
         payload: Dict[str, Any] = {"inputs": text}
         if parameters is not None:
             payload["parameters"] = parameters
         response = await self.post(json=payload, model=model, task="summarization")
-        return _bytes_to_dict(response)[0]["summary_text"]
+        return SummarizationOutput.parse_obj_as_list(response)[0]
 
     async def table_question_answering(
         self, table: Dict[str, Any], query: str, *, model: Optional[str] = None
-    ) -> TableQuestionAnsweringOutput:
+    ) -> TableQuestionAnsweringOutputElement:
         """
         Retrieve the answer to a question from information given in a table.
 
@@ -1097,7 +1055,7 @@ class AsyncInferenceClient:
                 Hub or a URL to a deployed Inference Endpoint.
 
         Returns:
-            `Dict`: a dictionary of table question answering output containing the answer, coordinates, cells and the aggregator used.
+            [`TableQuestionAnsweringOutputElement`]: a table question answering output containing the answer, coordinates, cells and the aggregator used.
 
         Raises:
             [`InferenceTimeoutError`]:
@@ -1113,7 +1071,7 @@ class AsyncInferenceClient:
         >>> query = "How many stars does the transformers repository have?"
         >>> table = {"Repository": ["Transformers", "Datasets", "Tokenizers"], "Stars": ["36542", "4512", "3934"]}
         >>> await client.table_question_answering(table, query, model="google/tapas-base-finetuned-wtq")
-        {'answer': 'AVERAGE > 36542', 'coordinates': [[0, 1]], 'cells': ['36542'], 'aggregator': 'AVERAGE'}
+        TableQuestionAnsweringOutputElement(answer='36542', coordinates=[[0, 1]], cells=['36542'], aggregator='AVERAGE')
         ```
         """
         response = await self.post(
@@ -1124,7 +1082,7 @@ class AsyncInferenceClient:
             model=model,
             task="table-question-answering",
         )
-        return _bytes_to_dict(response)  # type: ignore
+        return TableQuestionAnsweringOutputElement.parse_obj_as_instance(response)
 
     async def tabular_classification(self, table: Dict[str, Any], *, model: Optional[str] = None) -> List[str]:
         """
@@ -1213,7 +1171,9 @@ class AsyncInferenceClient:
         response = await self.post(json={"table": table}, model=model, task="tabular-regression")
         return _bytes_to_list(response)
 
-    async def text_classification(self, text: str, *, model: Optional[str] = None) -> List[ClassificationOutput]:
+    async def text_classification(
+        self, text: str, *, model: Optional[str] = None
+    ) -> List[TextClassificationOutputElement]:
         """
         Perform text classification (e.g. sentiment-analysis) on the given text.
 
@@ -1226,7 +1186,7 @@ class AsyncInferenceClient:
                 Defaults to None.
 
         Returns:
-            `List[Dict]`: a list of dictionaries containing the predicted label and associated probability.
+            `List[TextClassificationOutputElement]`: a list of [`TextClassificationOutputElement`] items containing the predicted label and associated probability.
 
         Raises:
             [`InferenceTimeoutError`]:
@@ -1240,11 +1200,14 @@ class AsyncInferenceClient:
         >>> from huggingface_hub import AsyncInferenceClient
         >>> client = AsyncInferenceClient()
         >>> await client.text_classification("I like you")
-        [{'label': 'POSITIVE', 'score': 0.9998695850372314}, {'label': 'NEGATIVE', 'score': 0.0001304351753788069}]
+        [
+            TextClassificationOutputElement(label='POSITIVE', score=0.9998695850372314),
+            TextClassificationOutputElement(label='NEGATIVE', score=0.0001304351753788069),
+        ]
         ```
         """
         response = await self.post(json={"inputs": text}, model=model, task="text-classification")
-        return _bytes_to_list(response)[0]
+        return TextClassificationOutputElement.parse_obj_as_list(response)[0]  # type: ignore [return-value]
 
     @overload
     async def text_generation(  # type: ignore
@@ -1725,7 +1688,9 @@ class AsyncInferenceClient:
         """
         return await self.post(json={"inputs": text}, model=model, task="text-to-speech")
 
-    async def token_classification(self, text: str, *, model: Optional[str] = None) -> List[TokenClassificationOutput]:
+    async def token_classification(
+        self, text: str, *, model: Optional[str] = None
+    ) -> List[TokenClassificationOutputElement]:
         """
         Perform token classification on the given text.
         Usually used for sentence parsing, either grammatical, or Named Entity Recognition (NER) to understand keywords contained within text.
@@ -1739,7 +1704,7 @@ class AsyncInferenceClient:
                 Defaults to None.
 
         Returns:
-            `List[Dict]`: List of token classification outputs containing the entity group, confidence score, word, start and end index.
+            `List[TokenClassificationOutputElement]`: List of [`TokenClassificationOutputElement`] items containing the entity group, confidence score, word, start and end index.
 
         Raises:
             [`InferenceTimeoutError`]:
@@ -1753,16 +1718,22 @@ class AsyncInferenceClient:
         >>> from huggingface_hub import AsyncInferenceClient
         >>> client = AsyncInferenceClient()
         >>> await client.token_classification("My name is Sarah Jessica Parker but you can call me Jessica")
-        [{'entity_group': 'PER',
-        'score': 0.9971321225166321,
-        'word': 'Sarah Jessica Parker',
-        'start': 11,
-        'end': 31},
-        {'entity_group': 'PER',
-        'score': 0.9773476123809814,
-        'word': 'Jessica',
-        'start': 52,
-        'end': 59}]
+        [
+            TokenClassificationOutputElement(
+                entity_group='PER',
+                score=0.9971321225166321,
+                word='Sarah Jessica Parker',
+                start=11,
+                end=31,
+            ),
+            TokenClassificationOutputElement(
+                entity_group='PER',
+                score=0.9773476123809814,
+                word='Jessica',
+                start=52,
+                end=59,
+            )
+        ]
         ```
         """
         payload: Dict[str, Any] = {"inputs": text}
@@ -1771,11 +1742,11 @@ class AsyncInferenceClient:
             model=model,
             task="token-classification",
         )
-        return _bytes_to_list(response)
+        return TokenClassificationOutputElement.parse_obj_as_list(response)
 
     async def translation(
         self, text: str, *, model: Optional[str] = None, src_lang: Optional[str] = None, tgt_lang: Optional[str] = None
-    ) -> str:
+    ) -> TranslationOutput:
         """
         Convert text from one language to another.
 
@@ -1798,7 +1769,7 @@ class AsyncInferenceClient:
                 Target language of the translation task, i.e. output language. Cannot be passed without `src_lang`.
 
         Returns:
-            `str`: The generated translated text.
+            [`TranslationOutput`]: The generated translated text.
 
         Raises:
             [`InferenceTimeoutError`]:
@@ -1816,7 +1787,7 @@ class AsyncInferenceClient:
         >>> await client.translation("My name is Wolfgang and I live in Berlin")
         'Mein Name ist Wolfgang und ich lebe in Berlin.'
         >>> await client.translation("My name is Wolfgang and I live in Berlin", model="Helsinki-NLP/opus-mt-en-fr")
-        "Je m'appelle Wolfgang et je vis à Berlin."
+        TranslationOutput(translation_text='Je m\'appelle Wolfgang et je vis à Berlin.')
         ```
 
         Specifying languages:
@@ -1837,11 +1808,59 @@ class AsyncInferenceClient:
         if src_lang and tgt_lang:
             payload["parameters"] = {"src_lang": src_lang, "tgt_lang": tgt_lang}
         response = await self.post(json=payload, model=model, task="translation")
-        return _bytes_to_dict(response)[0]["translation_text"]
+        return TranslationOutput.parse_obj_as_list(response)[0]
+
+    async def visual_question_answering(
+        self,
+        image: ContentT,
+        question: str,
+        *,
+        model: Optional[str] = None,
+    ) -> List[VisualQuestionAnsweringOutputElement]:
+        """
+        Answering open-ended questions based on an image.
+
+        Args:
+            image (`Union[str, Path, bytes, BinaryIO]`):
+                The input image for the context. It can be raw bytes, an image file, or a URL to an online image.
+            question (`str`):
+                Question to be answered.
+            model (`str`, *optional*):
+                The model to use for the visual question answering task. Can be a model ID hosted on the Hugging Face Hub or a URL to
+                a deployed Inference Endpoint. If not provided, the default recommended visual question answering model will be used.
+                Defaults to None.
+
+        Returns:
+            `List[VisualQuestionAnsweringOutputElement]`: a list of [`VisualQuestionAnsweringOutputElement`] items containing the predicted label and associated probability.
+
+        Raises:
+            `InferenceTimeoutError`:
+                If the model is unavailable or the request times out.
+            `aiohttp.ClientResponseError`:
+                If the request fails with an HTTP error status code other than HTTP 503.
+
+        Example:
+        ```py
+        # Must be run in an async context
+        >>> from huggingface_hub import AsyncInferenceClient
+        >>> client = AsyncInferenceClient()
+        >>> await client.visual_question_answering(
+        ...     image="https://huggingface.co/datasets/mishig/sample_images/resolve/main/tiger.jpg",
+        ...     question="What is the animal doing?"
+        ... )
+        [
+            VisualQuestionAnsweringOutputElement(score=0.778609573841095, answer='laying down'),
+            VisualQuestionAnsweringOutputElement(score=0.6957435607910156, answer='sitting'),
+        ]
+        ```
+        """
+        payload: Dict[str, Any] = {"question": question, "image": _b64_encode(image)}
+        response = await self.post(json=payload, model=model, task="visual-question-answering")
+        return VisualQuestionAnsweringOutputElement.parse_obj_as_list(response)
 
     async def zero_shot_classification(
         self, text: str, labels: List[str], *, multi_label: bool = False, model: Optional[str] = None
-    ) -> List[ClassificationOutput]:
+    ) -> List[ZeroShotClassificationOutputElement]:
         """
         Provide as input a text and a set of candidate labels to classify the input text.
 
@@ -1857,7 +1876,7 @@ class AsyncInferenceClient:
                 Inference Endpoint. This parameter overrides the model defined at the instance level. Defaults to None.
 
         Returns:
-            `List[Dict]`: List of classification outputs containing the predicted labels and their confidence.
+            `List[ZeroShotClassificationOutputElement]`: List of [`ZeroShotClassificationOutputElement`] items containing the predicted labels and their confidence.
 
         Raises:
             [`InferenceTimeoutError`]:
@@ -1878,19 +1897,19 @@ class AsyncInferenceClient:
         >>> labels = ["space & cosmos", "scientific discovery", "microbiology", "robots", "archeology"]
         >>> await client.zero_shot_classification(text, labels)
         [
-            {"label": "scientific discovery", "score": 0.7961668968200684},
-            {"label": "space & cosmos", "score": 0.18570658564567566},
-            {"label": "microbiology", "score": 0.00730885099619627},
-            {"label": "archeology", "score": 0.006258360575884581},
-            {"label": "robots", "score": 0.004559356719255447},
+            ZeroShotClassificationOutputElement(label='scientific discovery', score=0.7961668968200684),
+            ZeroShotClassificationOutputElement(label='space & cosmos', score=0.18570658564567566),
+            ZeroShotClassificationOutputElement(label='microbiology', score=0.00730885099619627),
+            ZeroShotClassificationOutputElement(label='archeology', score=0.006258360575884581),
+            ZeroShotClassificationOutputElement(label='robots', score=0.004559356719255447),
         ]
         >>> await client.zero_shot_classification(text, labels, multi_label=True)
         [
-            {"label": "scientific discovery", "score": 0.9829297661781311},
-            {"label": "space & cosmos", "score": 0.755190908908844},
-            {"label": "microbiology", "score": 0.0005462635890580714},
-            {"label": "archeology", "score": 0.00047131875180639327},
-            {"label": "robots", "score": 0.00030448526376858354},
+            ZeroShotClassificationOutputElement(label='scientific discovery', score=0.9829297661781311),
+            ZeroShotClassificationOutputElement(label='space & cosmos', score=0.755190908908844),
+            ZeroShotClassificationOutputElement(label='microbiology', score=0.0005462635890580714),
+            ZeroShotClassificationOutputElement(label='archeology', score=0.00047131875180639327),
+            ZeroShotClassificationOutputElement(label='robots', score=0.00030448526376858354),
         ]
         ```
         """
@@ -1910,11 +1929,14 @@ class AsyncInferenceClient:
             task="zero-shot-classification",
         )
         output = _bytes_to_dict(response)
-        return [{"label": label, "score": score} for label, score in zip(output["labels"], output["scores"])]
+        return [
+            ZeroShotClassificationOutputElement.parse_obj_as_instance({"label": label, "score": score})
+            for label, score in zip(output["labels"], output["scores"])
+        ]
 
     async def zero_shot_image_classification(
         self, image: ContentT, labels: List[str], *, model: Optional[str] = None
-    ) -> List[ClassificationOutput]:
+    ) -> List[ZeroShotImageClassificationOutputElement]:
         """
         Provide input image and text labels to predict text labels for the image.
 
@@ -1928,7 +1950,7 @@ class AsyncInferenceClient:
                 Inference Endpoint. This parameter overrides the model defined at the instance level. Defaults to None.
 
         Returns:
-            `List[Dict]`: List of classification outputs containing the predicted labels and their confidence.
+            `List[ZeroShotImageClassificationOutputElement]`: List of [`ZeroShotImageClassificationOutputElement`] items containing the predicted labels and their confidence.
 
         Raises:
             [`InferenceTimeoutError`]:
@@ -1946,7 +1968,7 @@ class AsyncInferenceClient:
         ...     "https://upload.wikimedia.org/wikipedia/commons/thumb/4/43/Cute_dog.jpg/320px-Cute_dog.jpg",
         ...     labels=["dog", "cat", "horse"],
         ... )
-        [{"label": "dog", "score": 0.956}, ...]
+        [ZeroShotImageClassificationOutputElement(label='dog', score=0.956),...]
         ```
         """
         # Raise ValueError if input is less than 2 labels
@@ -1958,7 +1980,7 @@ class AsyncInferenceClient:
             model=model,
             task="zero-shot-image-classification",
         )
-        return _bytes_to_list(response)
+        return ZeroShotImageClassificationOutputElement.parse_obj_as_list(response)
 
     def _resolve_url(self, model: Optional[str] = None, task: Optional[str] = None) -> str:
         model = model or self.model
diff --git a/src/huggingface_hub/inference/_generated/types/__init__.py b/src/huggingface_hub/inference/_generated/types/__init__.py
new file mode 100644
index 00000000..6c766768
--- /dev/null
+++ b/src/huggingface_hub/inference/_generated/types/__init__.py
@@ -0,0 +1,103 @@
+# This file is auto-generated by `utils/generate_inference_types.py`.
+# Do not modify it manually.
+#
+# ruff: noqa: F401
+
+from .audio_classification import (
+    AudioClassificationInput,
+    AudioClassificationOutputElement,
+    AudioClassificationParameters,
+)
+from .audio_to_audio import AudioToAudioInput, AudioToAudioOutputElement
+from .automatic_speech_recognition import (
+    AutomaticSpeechRecognitionInput,
+    AutomaticSpeechRecognitionOutput,
+    AutomaticSpeechRecognitionOutputChunk,
+    AutomaticSpeechRecognitionParameters,
+    GenerationParameters,
+)
+from .base import BaseInferenceType
+from .depth_estimation import DepthEstimationInput, DepthEstimationOutput
+from .document_question_answering import (
+    DocumentQuestionAnsweringInput,
+    DocumentQuestionAnsweringInputData,
+    DocumentQuestionAnsweringOutputElement,
+    DocumentQuestionAnsweringParameters,
+)
+from .feature_extraction import FeatureExtractionInput
+from .fill_mask import FillMaskInput, FillMaskOutputElement, FillMaskParameters
+from .image_classification import (
+    ImageClassificationInput,
+    ImageClassificationOutputElement,
+    ImageClassificationParameters,
+)
+from .image_segmentation import ImageSegmentationInput, ImageSegmentationOutputElement, ImageSegmentationParameters
+from .image_to_image import ImageToImageInput, ImageToImageOutput, ImageToImageParameters, TargetSize
+from .image_to_text import ImageToTextInput, ImageToTextOutput, ImageToTextParameters
+from .object_detection import (
+    BoundingBox,
+    ObjectDetectionInput,
+    ObjectDetectionOutputElement,
+    ObjectDetectionParameters,
+)
+from .question_answering import (
+    QuestionAnsweringInput,
+    QuestionAnsweringInputData,
+    QuestionAnsweringOutputElement,
+    QuestionAnsweringParameters,
+)
+from .sentence_similarity import SentenceSimilarityInput, SentenceSimilarityInputData
+from .summarization import SummarizationInput, SummarizationOutput, Text2TextGenerationParameters
+from .table_question_answering import (
+    TableQuestionAnsweringInput,
+    TableQuestionAnsweringInputData,
+    TableQuestionAnsweringOutputElement,
+)
+from .text2text_generation import Text2TextGenerationInput, Text2TextGenerationOutput
+from .text_classification import TextClassificationInput, TextClassificationOutputElement, TextClassificationParameters
+from .text_generation import (
+    PrefillToken,
+    TextGenerationInput,
+    TextGenerationOutput,
+    TextGenerationOutputDetails,
+    TextGenerationParameters,
+    TextGenerationSequenceDetails,
+    Token,
+)
+from .text_to_audio import TextToAudioInput, TextToAudioOutput, TextToAudioParameters
+from .text_to_image import TextToImageInput, TextToImageOutput, TextToImageParameters
+from .text_to_speech import TextToSpeechInput, TextToSpeechOutput
+from .token_classification import (
+    TokenClassificationInput,
+    TokenClassificationOutputElement,
+    TokenClassificationParameters,
+)
+from .translation import TranslationInput, TranslationOutput
+from .video_classification import (
+    VideoClassificationInput,
+    VideoClassificationOutputElement,
+    VideoClassificationParameters,
+)
+from .visual_question_answering import (
+    VisualQuestionAnsweringInput,
+    VisualQuestionAnsweringInputData,
+    VisualQuestionAnsweringOutputElement,
+    VisualQuestionAnsweringParameters,
+)
+from .zero_shot_classification import (
+    ZeroShotClassificationInput,
+    ZeroShotClassificationInputData,
+    ZeroShotClassificationOutputElement,
+    ZeroShotClassificationParameters,
+)
+from .zero_shot_image_classification import (
+    ZeroShotImageClassificationInput,
+    ZeroShotImageClassificationInputData,
+    ZeroShotImageClassificationOutputElement,
+    ZeroShotImageClassificationParameters,
+)
+from .zero_shot_object_detection import (
+    ZeroShotObjectDetectionInput,
+    ZeroShotObjectDetectionInputData,
+    ZeroShotObjectDetectionOutputElement,
+)
diff --git a/src/huggingface_hub/inference/_generated/types/audio_classification.py b/src/huggingface_hub/inference/_generated/types/audio_classification.py
new file mode 100644
index 00000000..451ba9e9
--- /dev/null
+++ b/src/huggingface_hub/inference/_generated/types/audio_classification.py
@@ -0,0 +1,43 @@
+# Inference code generated from the JSON schema spec in @huggingface/tasks.
+#
+# See:
+#   - script: https://github.com/huggingface/huggingface.js/blob/main/packages/tasks/scripts/inference-codegen.ts
+#   - specs:  https://github.com/huggingface/huggingface.js/tree/main/packages/tasks/src/tasks.
+from dataclasses import dataclass
+from typing import Any, Literal, Optional
+
+from .base import BaseInferenceType
+
+
+ClassificationOutputTransform = Literal["sigmoid", "softmax", "none"]
+
+
+@dataclass
+class AudioClassificationParameters(BaseInferenceType):
+    """Additional inference parameters
+    Additional inference parameters for Audio Classification
+    """
+
+    function_to_apply: Optional["ClassificationOutputTransform"]
+    top_k: Optional[int]
+    """When specified, limits the output to the top K most probable classes."""
+
+
+@dataclass
+class AudioClassificationInput(BaseInferenceType):
+    """Inputs for Audio Classification inference"""
+
+    inputs: Any
+    """The input audio data"""
+    parameters: Optional[AudioClassificationParameters]
+    """Additional inference parameters"""
+
+
+@dataclass
+class AudioClassificationOutputElement(BaseInferenceType):
+    """Outputs for Audio Classification inference"""
+
+    label: str
+    """The predicted class label."""
+    score: float
+    """The corresponding probability."""
diff --git a/src/huggingface_hub/inference/_generated/types/audio_to_audio.py b/src/huggingface_hub/inference/_generated/types/audio_to_audio.py
new file mode 100644
index 00000000..4f473ed1
--- /dev/null
+++ b/src/huggingface_hub/inference/_generated/types/audio_to_audio.py
@@ -0,0 +1,31 @@
+# Inference code generated from the JSON schema spec in @huggingface/tasks.
+#
+# See:
+#   - script: https://github.com/huggingface/huggingface.js/blob/main/packages/tasks/scripts/inference-codegen.ts
+#   - specs:  https://github.com/huggingface/huggingface.js/tree/main/packages/tasks/src/tasks.
+from dataclasses import dataclass
+from typing import Any
+
+from .base import BaseInferenceType
+
+
+@dataclass
+class AudioToAudioInput(BaseInferenceType):
+    """Inputs for Audio to Audio inference"""
+
+    inputs: Any
+    """The input audio data"""
+
+
+@dataclass
+class AudioToAudioOutputElement(BaseInferenceType):
+    """Outputs of inference for the Audio To Audio task
+    A generated audio file with its label.
+    """
+
+    blob: Any
+    """The generated audio file."""
+    content_type: str
+    """The content type of audio file."""
+    label: str
+    """The label of the audio file."""
diff --git a/src/huggingface_hub/inference/_generated/types/automatic_speech_recognition.py b/src/huggingface_hub/inference/_generated/types/automatic_speech_recognition.py
new file mode 100644
index 00000000..9b424ff6
--- /dev/null
+++ b/src/huggingface_hub/inference/_generated/types/automatic_speech_recognition.py
@@ -0,0 +1,116 @@
+# Inference code generated from the JSON schema spec in @huggingface/tasks.
+#
+# See:
+#   - script: https://github.com/huggingface/huggingface.js/blob/main/packages/tasks/scripts/inference-codegen.ts
+#   - specs:  https://github.com/huggingface/huggingface.js/tree/main/packages/tasks/src/tasks.
+from dataclasses import dataclass
+from typing import Any, List, Literal, Optional, Union
+
+from .base import BaseInferenceType
+
+
+EarlyStoppingEnum = Literal["never"]
+
+
+@dataclass
+class GenerationParameters(BaseInferenceType):
+    """Parametrization of the text generation process
+    Ad-hoc parametrization of the text generation process
+    """
+
+    do_sample: Optional[bool]
+    """Whether to use sampling instead of greedy decoding when generating new tokens."""
+    early_stopping: Optional[Union[bool, "EarlyStoppingEnum"]]
+    """Controls the stopping condition for beam-based methods."""
+    epsilon_cutoff: Optional[float]
+    """If set to float strictly between 0 and 1, only tokens with a conditional probability
+    greater than epsilon_cutoff will be sampled. In the paper, suggested values range from
+    3e-4 to 9e-4, depending on the size of the model. See [Truncation Sampling as Language
+    Model Desmoothing](https://hf.co/papers/2210.15191) for more details.
+    """
+    eta_cutoff: Optional[float]
+    """Eta sampling is a hybrid of locally typical sampling and epsilon sampling. If set to
+    float strictly between 0 and 1, a token is only considered if it is greater than either
+    eta_cutoff or sqrt(eta_cutoff) * exp(-entropy(softmax(next_token_logits))). The latter
+    term is intuitively the expected next token probability, scaled by sqrt(eta_cutoff). In
+    the paper, suggested values range from 3e-4 to 2e-3, depending on the size of the model.
+    See [Truncation Sampling as Language Model Desmoothing](https://hf.co/papers/2210.15191)
+    for more details.
+    """
+    max_length: Optional[int]
+    """The maximum length (in tokens) of the generated text, including the input."""
+    max_new_tokens: Optional[int]
+    """The maximum number of tokens to generate. Takes precedence over maxLength."""
+    min_length: Optional[int]
+    """The minimum length (in tokens) of the generated text, including the input."""
+    min_new_tokens: Optional[int]
+    """The minimum number of tokens to generate. Takes precedence over maxLength."""
+    num_beam_groups: Optional[int]
+    """Number of groups to divide num_beams into in order to ensure diversity among different
+    groups of beams. See [this paper](https://hf.co/papers/1610.02424) for more details.
+    """
+    num_beams: Optional[int]
+    """Number of beams to use for beam search."""
+    penalty_alpha: Optional[float]
+    """The value balances the model confidence and the degeneration penalty in contrastive
+    search decoding.
+    """
+    temperature: Optional[float]
+    """The value used to modulate the next token probabilities."""
+    top_k: Optional[int]
+    """The number of highest probability vocabulary tokens to keep for top-k-filtering."""
+    top_p: Optional[float]
+    """If set to float < 1, only the smallest set of most probable tokens with probabilities
+    that add up to top_p or higher are kept for generation.
+    """
+    typical_p: Optional[float]
+    """Local typicality measures how similar the conditional probability of predicting a target
+    token next is to the expected conditional probability of predicting a random token next,
+    given the partial text already generated. If set to float < 1, the smallest set of the
+    most locally typical tokens with probabilities that add up to typical_p or higher are
+    kept for generation. See [this paper](https://hf.co/papers/2202.00666) for more details.
+    """
+    use_cache: Optional[bool]
+    """Whether the model should use the past last key/values attentions to speed up decoding"""
+
+
+@dataclass
+class AutomaticSpeechRecognitionParameters(BaseInferenceType):
+    """Additional inference parameters
+    Additional inference parameters for Automatic Speech Recognition
+    """
+
+    generate: Optional[GenerationParameters]
+    """Parametrization of the text generation process"""
+    return_timestamps: Optional[bool]
+    """Whether to output corresponding timestamps with the generated text"""
+
+
+@dataclass
+class AutomaticSpeechRecognitionInput(BaseInferenceType):
+    """Inputs for Automatic Speech Recognition inference"""
+
+    inputs: Any
+    """The input audio data"""
+    parameters: Optional[AutomaticSpeechRecognitionParameters]
+    """Additional inference parameters"""
+
+
+@dataclass
+class AutomaticSpeechRecognitionOutputChunk(BaseInferenceType):
+    text: str
+    """A chunk of text identified by the model"""
+    timestamps: List[float]
+    """The start and end timestamps corresponding with the text"""
+
+
+@dataclass
+class AutomaticSpeechRecognitionOutput(BaseInferenceType):
+    """Outputs of inference for the Automatic Speech Recognition task"""
+
+    text: str
+    """The recognized text."""
+    chunks: Optional[List[AutomaticSpeechRecognitionOutputChunk]]
+    """When returnTimestamps is enabled, chunks contains a list of audio chunks identified by
+    the model.
+    """
diff --git a/src/huggingface_hub/inference/_generated/types/base.py b/src/huggingface_hub/inference/_generated/types/base.py
new file mode 100644
index 00000000..689fa08e
--- /dev/null
+++ b/src/huggingface_hub/inference/_generated/types/base.py
@@ -0,0 +1,148 @@
+# Copyright 2024 The HuggingFace Team. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""Contains a base class for all inference types."""
+import inspect
+import json
+import warnings
+from dataclasses import asdict, dataclass
+from typing import Any, Dict, List, Type, TypeVar, Union, get_args
+
+
+T = TypeVar("T", bound="BaseInferenceType")
+
+
+@dataclass
+class BaseInferenceType(dict):
+    """Base class for all inference types.
+
+    Object is a dataclass and a dict for backward compatibility but plan is to remove the dict part in the future.
+
+    Handle parsing from dict, list and json strings in a permissive way to ensure future-compatibility (e.g. all fields
+    are made optional, and non-expected fields are added as dict attributes).
+    """
+
+    @classmethod
+    def parse_obj_as_list(cls: Type[T], data: Union[bytes, str, List, Dict]) -> List[T]:
+        """Alias to parse server response and return a single instance.
+
+        See `parse_obj` for more details.
+        """
+        output = cls.parse_obj(data)
+        if not isinstance(output, list):
+            raise ValueError(f"Invalid input data for {cls}. Expected a list, but got {type(output)}.")
+        return output
+
+    @classmethod
+    def parse_obj_as_instance(cls: Type[T], data: Union[bytes, str, List, Dict]) -> T:
+        """Alias to parse server response and return a single instance.
+
+        See `parse_obj` for more details.
+        """
+        output = cls.parse_obj(data)
+        if isinstance(output, list):
+            raise ValueError(f"Invalid input data for {cls}. Expected a single instance, but got a list.")
+        return output
+
+    @classmethod
+    def parse_obj(cls: Type[T], data: Union[bytes, str, List, Dict]) -> Union[List[T], T]:
+        """Parse server response as a dataclass or list of dataclasses.
+
+        To enable future-compatibility, we want to handle cases where the server return more fields than expected.
+        In such cases, we don't want to raise an error but still create the dataclass object. Remaining fields are
+        added as dict attributes.
+        """
+        # Parse server response (from bytes)
+        if isinstance(data, bytes):
+            data = data.decode()
+        if isinstance(data, str):
+            data = json.loads(data)
+
+        # If a list, parse each item individually
+        if isinstance(data, List):
+            return [cls.parse_obj(d) for d in data]  # type: ignore [misc]
+
+        # At this point, we expect a dict
+        if not isinstance(data, dict):
+            raise ValueError(f"Invalid data type: {type(data)}")
+
+        init_values = {}
+        other_values = {}
+        for key, value in data.items():
+            key = normalize_key(key)
+            if key in cls.__dataclass_fields__ and cls.__dataclass_fields__[key].init:
+                if isinstance(value, dict) or isinstance(value, list):
+                    field_type = cls.__dataclass_fields__[key].type
+
+                    # if `field_type` is a `BaseInferenceType`, parse it
+                    if inspect.isclass(field_type) and issubclass(field_type, BaseInferenceType):
+                        value = field_type.parse_obj(value)
+
+                    # otherwise, recursively parse nested dataclasses (if possible)
+                    # `get_args` returns handle Union and Optional for us
+                    else:
+                        expected_types = get_args(field_type)
+                        for expected_type in expected_types:
+                            if getattr(expected_type, "_name", None) == "List":
+                                expected_type = get_args(expected_type)[
+                                    0
+                                ]  # assume same type for all items in the list
+                            if inspect.isclass(expected_type) and issubclass(expected_type, BaseInferenceType):
+                                value = expected_type.parse_obj(value)
+                                break
+                init_values[key] = value
+            else:
+                other_values[key] = value
+
+        # Make all missing fields default to None
+        # => ensure that dataclass initialization will never fail even if the server does not return all fields.
+        for key in cls.__dataclass_fields__:
+            if key not in init_values:
+                init_values[key] = None
+
+        # Initialize dataclass with expected values
+        item = cls(**init_values)
+
+        # Add remaining fields as dict attributes
+        item.update(other_values)
+        return item
+
+    def __post_init__(self):
+        self.update(asdict(self))
+
+    def __setitem__(self, __key: Any, __value: Any) -> None:
+        # Hacky way to keep dataclass values in sync when dict is updated
+        super().__setitem__(__key, __value)
+        if __key in self.__dataclass_fields__ and getattr(self, __key, None) != __value:
+            self.__setattr__(__key, __value)
+        return
+
+    def __setattr__(self, __name: str, __value: Any) -> None:
+        # Hacky way to keep dict values is sync when dataclass is updated
+        super().__setattr__(__name, __value)
+        if self.get(__name) != __value:
+            self[__name] = __value
+        return
+
+    def __getitem__(self, __key: Any) -> Any:
+        warnings.warn(
+            f"Accessing '{self.__class__.__name__}' values through dict is deprecated and "
+            "will be removed from version '0.25'. Use dataclass attributes instead.",
+            FutureWarning,
+        )
+        return super().__getitem__(__key)
+
+
+def normalize_key(key: str) -> str:
+    # e.g "content-type" -> "content_type", "Accept" -> "accept"
+    return key.replace("-", "_").replace(" ", "_").lower()
diff --git a/src/huggingface_hub/inference/_generated/types/depth_estimation.py b/src/huggingface_hub/inference/_generated/types/depth_estimation.py
new file mode 100644
index 00000000..4e3aa0dd
--- /dev/null
+++ b/src/huggingface_hub/inference/_generated/types/depth_estimation.py
@@ -0,0 +1,29 @@
+# Inference code generated from the JSON schema spec in @huggingface/tasks.
+#
+# See:
+#   - script: https://github.com/huggingface/huggingface.js/blob/main/packages/tasks/scripts/inference-codegen.ts
+#   - specs:  https://github.com/huggingface/huggingface.js/tree/main/packages/tasks/src/tasks.
+from dataclasses import dataclass
+from typing import Any, Dict, Optional
+
+from .base import BaseInferenceType
+
+
+@dataclass
+class DepthEstimationInput(BaseInferenceType):
+    """Inputs for Depth Estimation inference"""
+
+    inputs: Any
+    """The input image data"""
+    parameters: Optional[Dict[str, Any]]
+    """Additional inference parameters"""
+
+
+@dataclass
+class DepthEstimationOutput(BaseInferenceType):
+    """Outputs of inference for the Depth Estimation task"""
+
+    depth: Any
+    """The predicted depth as an image"""
+    predicted_depth: Any
+    """The predicted depth as a tensor"""
diff --git a/src/huggingface_hub/inference/_generated/types/document_question_answering.py b/src/huggingface_hub/inference/_generated/types/document_question_answering.py
new file mode 100644
index 00000000..530d0032
--- /dev/null
+++ b/src/huggingface_hub/inference/_generated/types/document_question_answering.py
@@ -0,0 +1,85 @@
+# Inference code generated from the JSON schema spec in @huggingface/tasks.
+#
+# See:
+#   - script: https://github.com/huggingface/huggingface.js/blob/main/packages/tasks/scripts/inference-codegen.ts
+#   - specs:  https://github.com/huggingface/huggingface.js/tree/main/packages/tasks/src/tasks.
+from dataclasses import dataclass
+from typing import Any, List, Optional, Union
+
+from .base import BaseInferenceType
+
+
+@dataclass
+class DocumentQuestionAnsweringInputData(BaseInferenceType):
+    """One (document, question) pair to answer"""
+
+    image: Any
+    """The image on which the question is asked"""
+    question: str
+    """A question to ask of the document"""
+
+
+@dataclass
+class DocumentQuestionAnsweringParameters(BaseInferenceType):
+    """Additional inference parameters
+    Additional inference parameters for Document Question Answering
+    """
+
+    doc_stride: Optional[int]
+    """If the words in the document are too long to fit with the question for the model, it will
+    be split in several chunks with some overlap. This argument controls the size of that
+    overlap.
+    """
+    handle_impossible_answer: Optional[bool]
+    """Whether to accept impossible as an answer"""
+    lang: Optional[str]
+    """Language to use while running OCR. Defaults to english."""
+    max_answer_len: Optional[int]
+    """The maximum length of predicted answers (e.g., only answers with a shorter length are
+    considered).
+    """
+    max_question_len: Optional[int]
+    """The maximum length of the question after tokenization. It will be truncated if needed."""
+    max_seq_len: Optional[int]
+    """The maximum length of the total sentence (context + question) in tokens of each chunk
+    passed to the model. The context will be split in several chunks (using doc_stride as
+    overlap) if needed.
+    """
+    top_k: Optional[int]
+    """The number of answers to return (will be chosen by order of likelihood). Can return less
+    than top_k answers if there are not enough options available within the context.
+    """
+    word_boxes: Optional[List[Union[List[float], str]]]
+    """A list of words and bounding boxes (normalized 0->1000). If provided, the inference will
+    skip the OCR step and use the provided bounding boxes instead.
+    """
+
+
+@dataclass
+class DocumentQuestionAnsweringInput(BaseInferenceType):
+    """Inputs for Document Question Answering inference"""
+
+    inputs: DocumentQuestionAnsweringInputData
+    """One (document, question) pair to answer"""
+    parameters: Optional[DocumentQuestionAnsweringParameters]
+    """Additional inference parameters"""
+
+
+@dataclass
+class DocumentQuestionAnsweringOutputElement(BaseInferenceType):
+    """Outputs of inference for the Document Question Answering task"""
+
+    answer: str
+    """The answer to the question."""
+    end: int
+    """The end word index of the answer (in the OCR’d version of the input or provided word
+    boxes).
+    """
+    score: float
+    """The probability associated to the answer."""
+    start: int
+    """The start word index of the answer (in the OCR’d version of the input or provided word
+    boxes).
+    """
+    words: List[int]
+    """The index of each word/box pair that is in the answer"""
diff --git a/src/huggingface_hub/inference/_generated/types/feature_extraction.py b/src/huggingface_hub/inference/_generated/types/feature_extraction.py
new file mode 100644
index 00000000..f9a71bbc
--- /dev/null
+++ b/src/huggingface_hub/inference/_generated/types/feature_extraction.py
@@ -0,0 +1,19 @@
+# Inference code generated from the JSON schema spec in @huggingface/tasks.
+#
+# See:
+#   - script: https://github.com/huggingface/huggingface.js/blob/main/packages/tasks/scripts/inference-codegen.ts
+#   - specs:  https://github.com/huggingface/huggingface.js/tree/main/packages/tasks/src/tasks.
+from dataclasses import dataclass
+from typing import Any, Dict, Optional
+
+from .base import BaseInferenceType
+
+
+@dataclass
+class FeatureExtractionInput(BaseInferenceType):
+    """Inputs for Text Embedding inference"""
+
+    inputs: str
+    """The text to get the embeddings of"""
+    parameters: Optional[Dict[str, Any]]
+    """Additional inference parameters"""
diff --git a/src/huggingface_hub/inference/_generated/types/fill_mask.py b/src/huggingface_hub/inference/_generated/types/fill_mask.py
new file mode 100644
index 00000000..897caa43
--- /dev/null
+++ b/src/huggingface_hub/inference/_generated/types/fill_mask.py
@@ -0,0 +1,50 @@
+# Inference code generated from the JSON schema spec in @huggingface/tasks.
+#
+# See:
+#   - script: https://github.com/huggingface/huggingface.js/blob/main/packages/tasks/scripts/inference-codegen.ts
+#   - specs:  https://github.com/huggingface/huggingface.js/tree/main/packages/tasks/src/tasks.
+from dataclasses import dataclass
+from typing import Any, List, Optional
+
+from .base import BaseInferenceType
+
+
+@dataclass
+class FillMaskParameters(BaseInferenceType):
+    """Additional inference parameters
+    Additional inference parameters for Fill Mask
+    """
+
+    targets: Optional[List[str]]
+    """When passed, the model will limit the scores to the passed targets instead of looking up
+    in the whole vocabulary. If the provided targets are not in the model vocab, they will be
+    tokenized and the first resulting token will be used (with a warning, and that might be
+    slower).
+    """
+    top_k: Optional[int]
+    """When passed, overrides the number of predictions to return."""
+
+
+@dataclass
+class FillMaskInput(BaseInferenceType):
+    """Inputs for Fill Mask inference"""
+
+    inputs: str
+    """The text with masked tokens"""
+    parameters: Optional[FillMaskParameters]
+    """Additional inference parameters"""
+
+
+@dataclass
+class FillMaskOutputElement(BaseInferenceType):
+    """Outputs of inference for the Fill Mask task"""
+
+    score: float
+    """The corresponding probability"""
+    sequence: str
+    """The corresponding input with the mask token prediction."""
+    token: int
+    """The predicted token id (to replace the masked one)."""
+    token_str: Any
+    fill_mask_output_token_str: Optional[str]
+    """The predicted token (to replace the masked one)."""
diff --git a/src/huggingface_hub/inference/_generated/types/image_classification.py b/src/huggingface_hub/inference/_generated/types/image_classification.py
new file mode 100644
index 00000000..78740b2c
--- /dev/null
+++ b/src/huggingface_hub/inference/_generated/types/image_classification.py
@@ -0,0 +1,43 @@
+# Inference code generated from the JSON schema spec in @huggingface/tasks.
+#
+# See:
+#   - script: https://github.com/huggingface/huggingface.js/blob/main/packages/tasks/scripts/inference-codegen.ts
+#   - specs:  https://github.com/huggingface/huggingface.js/tree/main/packages/tasks/src/tasks.
+from dataclasses import dataclass
+from typing import Any, Literal, Optional
+
+from .base import BaseInferenceType
+
+
+ClassificationOutputTransform = Literal["sigmoid", "softmax", "none"]
+
+
+@dataclass
+class ImageClassificationParameters(BaseInferenceType):
+    """Additional inference parameters
+    Additional inference parameters for Image Classification
+    """
+
+    function_to_apply: Optional["ClassificationOutputTransform"]
+    top_k: Optional[int]
+    """When specified, limits the output to the top K most probable classes."""
+
+
+@dataclass
+class ImageClassificationInput(BaseInferenceType):
+    """Inputs for Image Classification inference"""
+
+    inputs: Any
+    """The input image data"""
+    parameters: Optional[ImageClassificationParameters]
+    """Additional inference parameters"""
+
+
+@dataclass
+class ImageClassificationOutputElement(BaseInferenceType):
+    """Outputs of inference for the Image Classification task"""
+
+    label: str
+    """The predicted class label."""
+    score: float
+    """The corresponding probability."""
diff --git a/src/huggingface_hub/inference/_generated/types/image_segmentation.py b/src/huggingface_hub/inference/_generated/types/image_segmentation.py
new file mode 100644
index 00000000..0c4a60cb
--- /dev/null
+++ b/src/huggingface_hub/inference/_generated/types/image_segmentation.py
@@ -0,0 +1,52 @@
+# Inference code generated from the JSON schema spec in @huggingface/tasks.
+#
+# See:
+#   - script: https://github.com/huggingface/huggingface.js/blob/main/packages/tasks/scripts/inference-codegen.ts
+#   - specs:  https://github.com/huggingface/huggingface.js/tree/main/packages/tasks/src/tasks.
+from dataclasses import dataclass
+from typing import Any, Literal, Optional
+
+from .base import BaseInferenceType
+
+
+ImageSegmentationSubtask = Literal["instance", "panoptic", "semantic"]
+
+
+@dataclass
+class ImageSegmentationParameters(BaseInferenceType):
+    """Additional inference parameters
+    Additional inference parameters for Image Segmentation
+    """
+
+    mask_threshold: Optional[float]
+    """Threshold to use when turning the predicted masks into binary values."""
+    overlap_mask_area_threshold: Optional[float]
+    """Mask overlap threshold to eliminate small, disconnected segments."""
+    subtask: Optional["ImageSegmentationSubtask"]
+    """Segmentation task to be performed, depending on model capabilities."""
+    threshold: Optional[float]
+    """Probability threshold to filter out predicted masks."""
+
+
+@dataclass
+class ImageSegmentationInput(BaseInferenceType):
+    """Inputs for Image Segmentation inference"""
+
+    inputs: Any
+    """The input image data"""
+    parameters: Optional[ImageSegmentationParameters]
+    """Additional inference parameters"""
+
+
+@dataclass
+class ImageSegmentationOutputElement(BaseInferenceType):
+    """Outputs of inference for the Image Segmentation task
+    A predicted mask / segment
+    """
+
+    label: str
+    """The label of the predicted segment"""
+    mask: Any
+    """The corresponding mask as a black-and-white image"""
+    score: Optional[float]
+    """The score or confidence degreee the model has"""
diff --git a/src/huggingface_hub/inference/_generated/types/image_to_image.py b/src/huggingface_hub/inference/_generated/types/image_to_image.py
new file mode 100644
index 00000000..469bd001
--- /dev/null
+++ b/src/huggingface_hub/inference/_generated/types/image_to_image.py
@@ -0,0 +1,55 @@
+# Inference code generated from the JSON schema spec in @huggingface/tasks.
+#
+# See:
+#   - script: https://github.com/huggingface/huggingface.js/blob/main/packages/tasks/scripts/inference-codegen.ts
+#   - specs:  https://github.com/huggingface/huggingface.js/tree/main/packages/tasks/src/tasks.
+from dataclasses import dataclass
+from typing import Any, List, Optional
+
+from .base import BaseInferenceType
+
+
+@dataclass
+class TargetSize(BaseInferenceType):
+    """The size in pixel of the output image"""
+
+    height: int
+    width: int
+
+
+@dataclass
+class ImageToImageParameters(BaseInferenceType):
+    """Additional inference parameters
+    Additional inference parameters for Image To Image
+    """
+
+    guidance_scale: Optional[float]
+    """For diffusion models. A higher guidance scale value encourages the model to generate
+    images closely linked to the text prompt at the expense of lower image quality.
+    """
+    negative_prompt: Optional[List[str]]
+    """One or several prompt to guide what NOT to include in image generation."""
+    num_inference_steps: Optional[int]
+    """For diffusion models. The number of denoising steps. More denoising steps usually lead to
+    a higher quality image at the expense of slower inference.
+    """
+    target_size: Optional[TargetSize]
+    """The size in pixel of the output image"""
+
+
+@dataclass
+class ImageToImageInput(BaseInferenceType):
+    """Inputs for Image To Image inference"""
+
+    inputs: Any
+    """The input image data"""
+    parameters: Optional[ImageToImageParameters]
+    """Additional inference parameters"""
+
+
+@dataclass
+class ImageToImageOutput(BaseInferenceType):
+    """Outputs of inference for the Image To Image task"""
+
+    image: Any
+    """The output image"""
diff --git a/src/huggingface_hub/inference/_generated/types/image_to_text.py b/src/huggingface_hub/inference/_generated/types/image_to_text.py
new file mode 100644
index 00000000..cee4a9a2
--- /dev/null
+++ b/src/huggingface_hub/inference/_generated/types/image_to_text.py
@@ -0,0 +1,105 @@
+# Inference code generated from the JSON schema spec in @huggingface/tasks.
+#
+# See:
+#   - script: https://github.com/huggingface/huggingface.js/blob/main/packages/tasks/scripts/inference-codegen.ts
+#   - specs:  https://github.com/huggingface/huggingface.js/tree/main/packages/tasks/src/tasks.
+from dataclasses import dataclass
+from typing import Any, Literal, Optional, Union
+
+from .base import BaseInferenceType
+
+
+EarlyStoppingEnum = Literal["never"]
+
+
+@dataclass
+class GenerationParameters(BaseInferenceType):
+    """Parametrization of the text generation process
+    Ad-hoc parametrization of the text generation process
+    """
+
+    do_sample: Optional[bool]
+    """Whether to use sampling instead of greedy decoding when generating new tokens."""
+    early_stopping: Optional[Union[bool, "EarlyStoppingEnum"]]
+    """Controls the stopping condition for beam-based methods."""
+    epsilon_cutoff: Optional[float]
+    """If set to float strictly between 0 and 1, only tokens with a conditional probability
+    greater than epsilon_cutoff will be sampled. In the paper, suggested values range from
+    3e-4 to 9e-4, depending on the size of the model. See [Truncation Sampling as Language
+    Model Desmoothing](https://hf.co/papers/2210.15191) for more details.
+    """
+    eta_cutoff: Optional[float]
+    """Eta sampling is a hybrid of locally typical sampling and epsilon sampling. If set to
+    float strictly between 0 and 1, a token is only considered if it is greater than either
+    eta_cutoff or sqrt(eta_cutoff) * exp(-entropy(softmax(next_token_logits))). The latter
+    term is intuitively the expected next token probability, scaled by sqrt(eta_cutoff). In
+    the paper, suggested values range from 3e-4 to 2e-3, depending on the size of the model.
+    See [Truncation Sampling as Language Model Desmoothing](https://hf.co/papers/2210.15191)
+    for more details.
+    """
+    max_length: Optional[int]
+    """The maximum length (in tokens) of the generated text, including the input."""
+    max_new_tokens: Optional[int]
+    """The maximum number of tokens to generate. Takes precedence over maxLength."""
+    min_length: Optional[int]
+    """The minimum length (in tokens) of the generated text, including the input."""
+    min_new_tokens: Optional[int]
+    """The minimum number of tokens to generate. Takes precedence over maxLength."""
+    num_beam_groups: Optional[int]
+    """Number of groups to divide num_beams into in order to ensure diversity among different
+    groups of beams. See [this paper](https://hf.co/papers/1610.02424) for more details.
+    """
+    num_beams: Optional[int]
+    """Number of beams to use for beam search."""
+    penalty_alpha: Optional[float]
+    """The value balances the model confidence and the degeneration penalty in contrastive
+    search decoding.
+    """
+    temperature: Optional[float]
+    """The value used to modulate the next token probabilities."""
+    top_k: Optional[int]
+    """The number of highest probability vocabulary tokens to keep for top-k-filtering."""
+    top_p: Optional[float]
+    """If set to float < 1, only the smallest set of most probable tokens with probabilities
+    that add up to top_p or higher are kept for generation.
+    """
+    typical_p: Optional[float]
+    """Local typicality measures how similar the conditional probability of predicting a target
+    token next is to the expected conditional probability of predicting a random token next,
+    given the partial text already generated. If set to float < 1, the smallest set of the
+    most locally typical tokens with probabilities that add up to typical_p or higher are
+    kept for generation. See [this paper](https://hf.co/papers/2202.00666) for more details.
+    """
+    use_cache: Optional[bool]
+    """Whether the model should use the past last key/values attentions to speed up decoding"""
+
+
+@dataclass
+class ImageToTextParameters(BaseInferenceType):
+    """Additional inference parameters
+    Additional inference parameters for Image To Text
+    """
+
+    generate: Optional[GenerationParameters]
+    """Parametrization of the text generation process"""
+    max_new_tokens: Optional[int]
+    """The amount of maximum tokens to generate."""
+
+
+@dataclass
+class ImageToTextInput(BaseInferenceType):
+    """Inputs for Image To Text inference"""
+
+    inputs: Any
+    """The input image data"""
+    parameters: Optional[ImageToTextParameters]
+    """Additional inference parameters"""
+
+
+@dataclass
+class ImageToTextOutput(BaseInferenceType):
+    """Outputs of inference for the Image To Text task"""
+
+    generated_text: Any
+    image_to_text_output_generated_text: Optional[str]
+    """The generated text."""
diff --git a/src/huggingface_hub/inference/_generated/types/object_detection.py b/src/huggingface_hub/inference/_generated/types/object_detection.py
new file mode 100644
index 00000000..e6c974f3
--- /dev/null
+++ b/src/huggingface_hub/inference/_generated/types/object_detection.py
@@ -0,0 +1,55 @@
+# Inference code generated from the JSON schema spec in @huggingface/tasks.
+#
+# See:
+#   - script: https://github.com/huggingface/huggingface.js/blob/main/packages/tasks/scripts/inference-codegen.ts
+#   - specs:  https://github.com/huggingface/huggingface.js/tree/main/packages/tasks/src/tasks.
+from dataclasses import dataclass
+from typing import Any, Optional
+
+from .base import BaseInferenceType
+
+
+@dataclass
+class ObjectDetectionParameters(BaseInferenceType):
+    """Additional inference parameters
+    Additional inference parameters for Object Detection
+    """
+
+    threshold: Optional[float]
+    """The probability necessary to make a prediction."""
+
+
+@dataclass
+class ObjectDetectionInput(BaseInferenceType):
+    """Inputs for Object Detection inference"""
+
+    inputs: Any
+    """The input image data"""
+    parameters: Optional[ObjectDetectionParameters]
+    """Additional inference parameters"""
+
+
+@dataclass
+class BoundingBox(BaseInferenceType):
+    """The predicted bounding box. Coordinates are relative to the top left corner of the input
+    image.
+    """
+
+    xmax: int
+    xmin: int
+    ymax: int
+    ymin: int
+
+
+@dataclass
+class ObjectDetectionOutputElement(BaseInferenceType):
+    """Outputs of inference for the Object Detection task"""
+
+    box: BoundingBox
+    """The predicted bounding box. Coordinates are relative to the top left corner of the input
+    image.
+    """
+    label: str
+    """The predicted label for the bounding box"""
+    score: float
+    """The associated score / probability"""
diff --git a/src/huggingface_hub/inference/_generated/types/question_answering.py b/src/huggingface_hub/inference/_generated/types/question_answering.py
new file mode 100644
index 00000000..59cb181d
--- /dev/null
+++ b/src/huggingface_hub/inference/_generated/types/question_answering.py
@@ -0,0 +1,77 @@
+# Inference code generated from the JSON schema spec in @huggingface/tasks.
+#
+# See:
+#   - script: https://github.com/huggingface/huggingface.js/blob/main/packages/tasks/scripts/inference-codegen.ts
+#   - specs:  https://github.com/huggingface/huggingface.js/tree/main/packages/tasks/src/tasks.
+from dataclasses import dataclass
+from typing import Optional
+
+from .base import BaseInferenceType
+
+
+@dataclass
+class QuestionAnsweringInputData(BaseInferenceType):
+    """One (context, question) pair to answer"""
+
+    context: str
+    """The context to be used for answering the question"""
+    question: str
+    """The question to be answered"""
+
+
+@dataclass
+class QuestionAnsweringParameters(BaseInferenceType):
+    """Additional inference parameters
+    Additional inference parameters for Question Answering
+    """
+
+    align_to_words: Optional[bool]
+    """Attempts to align the answer to real words. Improves quality on space separated
+    languages. Might hurt on non-space-separated languages (like Japanese or Chinese)
+    """
+    doc_stride: Optional[int]
+    """If the context is too long to fit with the question for the model, it will be split in
+    several chunks with some overlap. This argument controls the size of that overlap.
+    """
+    handle_impossible_answer: Optional[bool]
+    """Whether to accept impossible as an answer."""
+    max_answer_len: Optional[int]
+    """The maximum length of predicted answers (e.g., only answers with a shorter length are
+    considered).
+    """
+    max_question_len: Optional[int]
+    """The maximum length of the question after tokenization. It will be truncated if needed."""
+    max_seq_len: Optional[int]
+    """The maximum length of the total sentence (context + question) in tokens of each chunk
+    passed to the model. The context will be split in several chunks (using docStride as
+    overlap) if needed.
+    """
+    top_k: Optional[int]
+    """The number of answers to return (will be chosen by order of likelihood). Note that we
+    return less than topk answers if there are not enough options available within the
+    context.
+    """
+
+
+@dataclass
+class QuestionAnsweringInput(BaseInferenceType):
+    """Inputs for Question Answering inference"""
+
+    inputs: QuestionAnsweringInputData
+    """One (context, question) pair to answer"""
+    parameters: Optional[QuestionAnsweringParameters]
+    """Additional inference parameters"""
+
+
+@dataclass
+class QuestionAnsweringOutputElement(BaseInferenceType):
+    """Outputs of inference for the Question Answering task"""
+
+    answer: str
+    """The answer to the question."""
+    end: int
+    """The character position in the input where the answer ends."""
+    score: float
+    """The probability associated to the answer."""
+    start: int
+    """The character position in the input where the answer begins."""
diff --git a/src/huggingface_hub/inference/_generated/types/sentence_similarity.py b/src/huggingface_hub/inference/_generated/types/sentence_similarity.py
new file mode 100644
index 00000000..44f21c73
--- /dev/null
+++ b/src/huggingface_hub/inference/_generated/types/sentence_similarity.py
@@ -0,0 +1,28 @@
+# Inference code generated from the JSON schema spec in @huggingface/tasks.
+#
+# See:
+#   - script: https://github.com/huggingface/huggingface.js/blob/main/packages/tasks/scripts/inference-codegen.ts
+#   - specs:  https://github.com/huggingface/huggingface.js/tree/main/packages/tasks/src/tasks.
+from dataclasses import dataclass
+from typing import Any, Dict, List, Optional
+
+from .base import BaseInferenceType
+
+
+@dataclass
+class SentenceSimilarityInputData(BaseInferenceType):
+    sentences: List[str]
+    """A list of strings which will be compared against the source_sentence."""
+    source_sentence: str
+    """The string that you wish to compare the other strings with. This can be a phrase,
+    sentence, or longer passage, depending on the model being used.
+    """
+
+
+@dataclass
+class SentenceSimilarityInput(BaseInferenceType):
+    """Inputs for Sentence similarity inference"""
+
+    inputs: SentenceSimilarityInputData
+    parameters: Optional[Dict[str, Any]]
+    """Additional inference parameters"""
diff --git a/src/huggingface_hub/inference/_generated/types/summarization.py b/src/huggingface_hub/inference/_generated/types/summarization.py
new file mode 100644
index 00000000..0e6a0507
--- /dev/null
+++ b/src/huggingface_hub/inference/_generated/types/summarization.py
@@ -0,0 +1,49 @@
+# Inference code generated from the JSON schema spec in @huggingface/tasks.
+#
+# See:
+#   - script: https://github.com/huggingface/huggingface.js/blob/main/packages/tasks/scripts/inference-codegen.ts
+#   - specs:  https://github.com/huggingface/huggingface.js/tree/main/packages/tasks/src/tasks.
+from dataclasses import dataclass
+from typing import Any, Dict, Literal, Optional
+
+from .base import BaseInferenceType
+
+
+Text2TextGenerationTruncationStrategy = Literal["do_not_truncate", "longest_first", "only_first", "only_second"]
+
+
+@dataclass
+class Text2TextGenerationParameters(BaseInferenceType):
+    """Additional inference parameters
+    Additional inference parameters for Text2text Generation
+    """
+
+    clean_up_tokenization_spaces: Optional[bool]
+    """Whether to clean up the potential extra spaces in the text output."""
+    generate_parameters: Optional[Dict[str, Any]]
+    """Additional parametrization of the text generation algorithm"""
+    truncation: Optional["Text2TextGenerationTruncationStrategy"]
+    """The truncation strategy to use"""
+
+
+@dataclass
+class SummarizationInput(BaseInferenceType):
+    """Inputs for Summarization inference
+    Inputs for Text2text Generation inference
+    """
+
+    inputs: str
+    """The input text data"""
+    parameters: Optional[Text2TextGenerationParameters]
+    """Additional inference parameters"""
+
+
+@dataclass
+class SummarizationOutput(BaseInferenceType):
+    """Outputs for Summarization inference
+    Outputs of inference for the Text2text Generation task
+    """
+
+    summary_text: Any
+    summarization_output_summary_text: Optional[str]
+    """The generated text."""
diff --git a/src/huggingface_hub/inference/_generated/types/table_question_answering.py b/src/huggingface_hub/inference/_generated/types/table_question_answering.py
new file mode 100644
index 00000000..d74a0983
--- /dev/null
+++ b/src/huggingface_hub/inference/_generated/types/table_question_answering.py
@@ -0,0 +1,45 @@
+# Inference code generated from the JSON schema spec in @huggingface/tasks.
+#
+# See:
+#   - script: https://github.com/huggingface/huggingface.js/blob/main/packages/tasks/scripts/inference-codegen.ts
+#   - specs:  https://github.com/huggingface/huggingface.js/tree/main/packages/tasks/src/tasks.
+from dataclasses import dataclass
+from typing import Any, Dict, List, Optional
+
+from .base import BaseInferenceType
+
+
+@dataclass
+class TableQuestionAnsweringInputData(BaseInferenceType):
+    """One (table, question) pair to answer"""
+
+    question: str
+    """The question to be answered about the table"""
+    table: Dict[str, List[str]]
+    """The table to serve as context for the questions"""
+
+
+@dataclass
+class TableQuestionAnsweringInput(BaseInferenceType):
+    """Inputs for Table Question Answering inference"""
+
+    inputs: TableQuestionAnsweringInputData
+    """One (table, question) pair to answer"""
+    parameters: Optional[Dict[str, Any]]
+    """Additional inference parameters"""
+
+
+@dataclass
+class TableQuestionAnsweringOutputElement(BaseInferenceType):
+    """Outputs of inference for the Table Question Answering task"""
+
+    answer: str
+    """The answer of the question given the table. If there is an aggregator, the answer will be
+    preceded by `AGGREGATOR >`.
+    """
+    cells: List[str]
+    """List of strings made up of the answer cell values."""
+    coordinates: List[List[int]]
+    """Coordinates of the cells of the answers."""
+    aggregator: Optional[str]
+    """If the model has an aggregator, this returns the aggregator."""
diff --git a/src/huggingface_hub/inference/_generated/types/text2text_generation.py b/src/huggingface_hub/inference/_generated/types/text2text_generation.py
new file mode 100644
index 00000000..740f114a
--- /dev/null
+++ b/src/huggingface_hub/inference/_generated/types/text2text_generation.py
@@ -0,0 +1,45 @@
+# Inference code generated from the JSON schema spec in @huggingface/tasks.
+#
+# See:
+#   - script: https://github.com/huggingface/huggingface.js/blob/main/packages/tasks/scripts/inference-codegen.ts
+#   - specs:  https://github.com/huggingface/huggingface.js/tree/main/packages/tasks/src/tasks.
+from dataclasses import dataclass
+from typing import Any, Dict, Literal, Optional
+
+from .base import BaseInferenceType
+
+
+Text2TextGenerationTruncationStrategy = Literal["do_not_truncate", "longest_first", "only_first", "only_second"]
+
+
+@dataclass
+class Text2TextGenerationParameters(BaseInferenceType):
+    """Additional inference parameters
+    Additional inference parameters for Text2text Generation
+    """
+
+    clean_up_tokenization_spaces: Optional[bool]
+    """Whether to clean up the potential extra spaces in the text output."""
+    generate_parameters: Optional[Dict[str, Any]]
+    """Additional parametrization of the text generation algorithm"""
+    truncation: Optional["Text2TextGenerationTruncationStrategy"]
+    """The truncation strategy to use"""
+
+
+@dataclass
+class Text2TextGenerationInput(BaseInferenceType):
+    """Inputs for Text2text Generation inference"""
+
+    inputs: str
+    """The input text data"""
+    parameters: Optional[Text2TextGenerationParameters]
+    """Additional inference parameters"""
+
+
+@dataclass
+class Text2TextGenerationOutput(BaseInferenceType):
+    """Outputs of inference for the Text2text Generation task"""
+
+    generated_text: Any
+    text2_text_generation_output_generated_text: Optional[str]
+    """The generated text."""
diff --git a/src/huggingface_hub/inference/_generated/types/text_classification.py b/src/huggingface_hub/inference/_generated/types/text_classification.py
new file mode 100644
index 00000000..1a0875ef
--- /dev/null
+++ b/src/huggingface_hub/inference/_generated/types/text_classification.py
@@ -0,0 +1,43 @@
+# Inference code generated from the JSON schema spec in @huggingface/tasks.
+#
+# See:
+#   - script: https://github.com/huggingface/huggingface.js/blob/main/packages/tasks/scripts/inference-codegen.ts
+#   - specs:  https://github.com/huggingface/huggingface.js/tree/main/packages/tasks/src/tasks.
+from dataclasses import dataclass
+from typing import Literal, Optional
+
+from .base import BaseInferenceType
+
+
+ClassificationOutputTransform = Literal["sigmoid", "softmax", "none"]
+
+
+@dataclass
+class TextClassificationParameters(BaseInferenceType):
+    """Additional inference parameters
+    Additional inference parameters for Text Classification
+    """
+
+    function_to_apply: Optional["ClassificationOutputTransform"]
+    top_k: Optional[int]
+    """When specified, limits the output to the top K most probable classes."""
+
+
+@dataclass
+class TextClassificationInput(BaseInferenceType):
+    """Inputs for Text Classification inference"""
+
+    inputs: str
+    """The text to classify"""
+    parameters: Optional[TextClassificationParameters]
+    """Additional inference parameters"""
+
+
+@dataclass
+class TextClassificationOutputElement(BaseInferenceType):
+    """Outputs of inference for the Text Classification task"""
+
+    label: str
+    """The predicted class label."""
+    score: float
+    """The corresponding probability."""
diff --git a/src/huggingface_hub/inference/_generated/types/text_generation.py b/src/huggingface_hub/inference/_generated/types/text_generation.py
new file mode 100644
index 00000000..86edb31d
--- /dev/null
+++ b/src/huggingface_hub/inference/_generated/types/text_generation.py
@@ -0,0 +1,128 @@
+# Inference code generated from the JSON schema spec in @huggingface/tasks.
+#
+# See:
+#   - script: https://github.com/huggingface/huggingface.js/blob/main/packages/tasks/scripts/inference-codegen.ts
+#   - specs:  https://github.com/huggingface/huggingface.js/tree/main/packages/tasks/src/tasks.
+from dataclasses import dataclass
+from typing import List, Literal, Optional
+
+from .base import BaseInferenceType
+
+
+@dataclass
+class TextGenerationParameters(BaseInferenceType):
+    """Additional inference parameters
+    Additional inference parameters for Text Generation
+    """
+
+    best_of: Optional[int]
+    """The number of sampling queries to run. Only the best one (in terms of total logprob) will
+    be returned.
+    """
+    decoder_input_details: Optional[bool]
+    """Whether or not to output decoder input details"""
+    details: Optional[bool]
+    """Whether or not to output details"""
+    do_sample: Optional[bool]
+    """Whether to use logits sampling instead of greedy decoding when generating new tokens."""
+    max_new_tokens: Optional[int]
+    """The maximum number of tokens to generate."""
+    repetition_penalty: Optional[float]
+    """The parameter for repetition penalty. A value of 1.0 means no penalty. See [this
+    paper](https://hf.co/papers/1909.05858) for more details.
+    """
+    return_full_text: Optional[bool]
+    """Whether to prepend the prompt to the generated text."""
+    seed: Optional[int]
+    """The random sampling seed."""
+    stop_sequences: Optional[List[str]]
+    """Stop generating tokens if a member of `stop_sequences` is generated."""
+    temperature: Optional[float]
+    """The value used to modulate the logits distribution."""
+    top_k: Optional[int]
+    """The number of highest probability vocabulary tokens to keep for top-k-filtering."""
+    top_p: Optional[float]
+    """If set to < 1, only the smallest set of most probable tokens with probabilities that add
+    up to `top_p` or higher are kept for generation.
+    """
+    truncate: Optional[int]
+    """Truncate input tokens to the given size."""
+    typical_p: Optional[float]
+    """Typical Decoding mass. See [Typical Decoding for Natural Language
+    Generation](https://hf.co/papers/2202.00666) for more information
+    """
+    watermark: Optional[bool]
+    """Watermarking with [A Watermark for Large Language Models](https://hf.co/papers/2301.10226)"""
+
+
+@dataclass
+class TextGenerationInput(BaseInferenceType):
+    """Inputs for Text Generation inference"""
+
+    inputs: str
+    """The text to initialize generation with"""
+    parameters: Optional[TextGenerationParameters]
+    """Additional inference parameters"""
+
+
+FinishReason = Literal["length", "eos_token", "stop_sequence"]
+
+
+@dataclass
+class PrefillToken(BaseInferenceType):
+    id: int
+    logprob: float
+    text: str
+    """The text associated with that token"""
+
+
+@dataclass
+class Token(BaseInferenceType):
+    id: int
+    logprob: float
+    special: bool
+    """Whether or not that token is a special one"""
+    text: str
+    """The text associated with that token"""
+
+
+@dataclass
+class TextGenerationSequenceDetails(BaseInferenceType):
+    finish_reason: "FinishReason"
+    """The reason why the generation was stopped."""
+    generated_text: int
+    """The generated text"""
+    generated_tokens: int
+    """The number of generated tokens"""
+    prefill: List[PrefillToken]
+    tokens: List[Token]
+    """The generated tokens and associated details"""
+    seed: Optional[int]
+    """The random seed used for generation"""
+
+
+@dataclass
+class TextGenerationOutputDetails(BaseInferenceType):
+    """When enabled, details about the generation"""
+
+    finish_reason: "FinishReason"
+    """The reason why the generation was stopped."""
+    generated_tokens: int
+    """The number of generated tokens"""
+    prefill: List[PrefillToken]
+    tokens: List[Token]
+    """The generated tokens and associated details"""
+    best_of_sequences: Optional[List[TextGenerationSequenceDetails]]
+    """Details about additional sequences when best_of is provided"""
+    seed: Optional[int]
+    """The random seed used for generation"""
+
+
+@dataclass
+class TextGenerationOutput(BaseInferenceType):
+    """Outputs for Text Generation inference"""
+
+    generated_text: str
+    """The generated text"""
+    details: Optional[TextGenerationOutputDetails]
+    """When enabled, details about the generation"""
diff --git a/src/huggingface_hub/inference/_generated/types/text_to_audio.py b/src/huggingface_hub/inference/_generated/types/text_to_audio.py
new file mode 100644
index 00000000..77fcfe65
--- /dev/null
+++ b/src/huggingface_hub/inference/_generated/types/text_to_audio.py
@@ -0,0 +1,105 @@
+# Inference code generated from the JSON schema spec in @huggingface/tasks.
+#
+# See:
+#   - script: https://github.com/huggingface/huggingface.js/blob/main/packages/tasks/scripts/inference-codegen.ts
+#   - specs:  https://github.com/huggingface/huggingface.js/tree/main/packages/tasks/src/tasks.
+from dataclasses import dataclass
+from typing import Any, Literal, Optional, Union
+
+from .base import BaseInferenceType
+
+
+EarlyStoppingEnum = Literal["never"]
+
+
+@dataclass
+class GenerationParameters(BaseInferenceType):
+    """Parametrization of the text generation process
+    Ad-hoc parametrization of the text generation process
+    """
+
+    do_sample: Optional[bool]
+    """Whether to use sampling instead of greedy decoding when generating new tokens."""
+    early_stopping: Optional[Union[bool, "EarlyStoppingEnum"]]
+    """Controls the stopping condition for beam-based methods."""
+    epsilon_cutoff: Optional[float]
+    """If set to float strictly between 0 and 1, only tokens with a conditional probability
+    greater than epsilon_cutoff will be sampled. In the paper, suggested values range from
+    3e-4 to 9e-4, depending on the size of the model. See [Truncation Sampling as Language
+    Model Desmoothing](https://hf.co/papers/2210.15191) for more details.
+    """
+    eta_cutoff: Optional[float]
+    """Eta sampling is a hybrid of locally typical sampling and epsilon sampling. If set to
+    float strictly between 0 and 1, a token is only considered if it is greater than either
+    eta_cutoff or sqrt(eta_cutoff) * exp(-entropy(softmax(next_token_logits))). The latter
+    term is intuitively the expected next token probability, scaled by sqrt(eta_cutoff). In
+    the paper, suggested values range from 3e-4 to 2e-3, depending on the size of the model.
+    See [Truncation Sampling as Language Model Desmoothing](https://hf.co/papers/2210.15191)
+    for more details.
+    """
+    max_length: Optional[int]
+    """The maximum length (in tokens) of the generated text, including the input."""
+    max_new_tokens: Optional[int]
+    """The maximum number of tokens to generate. Takes precedence over maxLength."""
+    min_length: Optional[int]
+    """The minimum length (in tokens) of the generated text, including the input."""
+    min_new_tokens: Optional[int]
+    """The minimum number of tokens to generate. Takes precedence over maxLength."""
+    num_beam_groups: Optional[int]
+    """Number of groups to divide num_beams into in order to ensure diversity among different
+    groups of beams. See [this paper](https://hf.co/papers/1610.02424) for more details.
+    """
+    num_beams: Optional[int]
+    """Number of beams to use for beam search."""
+    penalty_alpha: Optional[float]
+    """The value balances the model confidence and the degeneration penalty in contrastive
+    search decoding.
+    """
+    temperature: Optional[float]
+    """The value used to modulate the next token probabilities."""
+    top_k: Optional[int]
+    """The number of highest probability vocabulary tokens to keep for top-k-filtering."""
+    top_p: Optional[float]
+    """If set to float < 1, only the smallest set of most probable tokens with probabilities
+    that add up to top_p or higher are kept for generation.
+    """
+    typical_p: Optional[float]
+    """Local typicality measures how similar the conditional probability of predicting a target
+    token next is to the expected conditional probability of predicting a random token next,
+    given the partial text already generated. If set to float < 1, the smallest set of the
+    most locally typical tokens with probabilities that add up to typical_p or higher are
+    kept for generation. See [this paper](https://hf.co/papers/2202.00666) for more details.
+    """
+    use_cache: Optional[bool]
+    """Whether the model should use the past last key/values attentions to speed up decoding"""
+
+
+@dataclass
+class TextToAudioParameters(BaseInferenceType):
+    """Additional inference parameters
+    Additional inference parameters for Text To Audio
+    """
+
+    generate: Optional[GenerationParameters]
+    """Parametrization of the text generation process"""
+
+
+@dataclass
+class TextToAudioInput(BaseInferenceType):
+    """Inputs for Text To Audio inference"""
+
+    inputs: str
+    """The input text data"""
+    parameters: Optional[TextToAudioParameters]
+    """Additional inference parameters"""
+
+
+@dataclass
+class TextToAudioOutput(BaseInferenceType):
+    """Outputs of inference for the Text To Audio task"""
+
+    audio: Any
+    """The generated audio waveform."""
+    sampling_rate: Any
+    text_to_audio_output_sampling_rate: Optional[float]
+    """The sampling rate of the generated audio waveform."""
diff --git a/src/huggingface_hub/inference/_generated/types/text_to_image.py b/src/huggingface_hub/inference/_generated/types/text_to_image.py
new file mode 100644
index 00000000..108dc9f9
--- /dev/null
+++ b/src/huggingface_hub/inference/_generated/types/text_to_image.py
@@ -0,0 +1,57 @@
+# Inference code generated from the JSON schema spec in @huggingface/tasks.
+#
+# See:
+#   - script: https://github.com/huggingface/huggingface.js/blob/main/packages/tasks/scripts/inference-codegen.ts
+#   - specs:  https://github.com/huggingface/huggingface.js/tree/main/packages/tasks/src/tasks.
+from dataclasses import dataclass
+from typing import Any, List, Optional
+
+from .base import BaseInferenceType
+
+
+@dataclass
+class TargetSize(BaseInferenceType):
+    """The size in pixel of the output image"""
+
+    height: int
+    width: int
+
+
+@dataclass
+class TextToImageParameters(BaseInferenceType):
+    """Additional inference parameters
+    Additional inference parameters for Text To Image
+    """
+
+    guidance_scale: Optional[float]
+    """For diffusion models. A higher guidance scale value encourages the model to generate
+    images closely linked to the text prompt at the expense of lower image quality.
+    """
+    negative_prompt: Optional[List[str]]
+    """One or several prompt to guide what NOT to include in image generation."""
+    num_inference_steps: Optional[int]
+    """For diffusion models. The number of denoising steps. More denoising steps usually lead to
+    a higher quality image at the expense of slower inference.
+    """
+    scheduler: Optional[str]
+    """For diffusion models. Override the scheduler with a compatible one"""
+    target_size: Optional[TargetSize]
+    """The size in pixel of the output image"""
+
+
+@dataclass
+class TextToImageInput(BaseInferenceType):
+    """Inputs for Text To Image inference"""
+
+    inputs: str
+    """The input text data (sometimes called "prompt\""""
+    parameters: Optional[TextToImageParameters]
+    """Additional inference parameters"""
+
+
+@dataclass
+class TextToImageOutput(BaseInferenceType):
+    """Outputs of inference for the Text To Image task"""
+
+    image: Any
+    """The generated image"""
diff --git a/src/huggingface_hub/inference/_generated/types/text_to_speech.py b/src/huggingface_hub/inference/_generated/types/text_to_speech.py
new file mode 100644
index 00000000..26038b0a
--- /dev/null
+++ b/src/huggingface_hub/inference/_generated/types/text_to_speech.py
@@ -0,0 +1,109 @@
+# Inference code generated from the JSON schema spec in @huggingface/tasks.
+#
+# See:
+#   - script: https://github.com/huggingface/huggingface.js/blob/main/packages/tasks/scripts/inference-codegen.ts
+#   - specs:  https://github.com/huggingface/huggingface.js/tree/main/packages/tasks/src/tasks.
+from dataclasses import dataclass
+from typing import Any, Literal, Optional, Union
+
+from .base import BaseInferenceType
+
+
+EarlyStoppingEnum = Literal["never"]
+
+
+@dataclass
+class GenerationParameters(BaseInferenceType):
+    """Parametrization of the text generation process
+    Ad-hoc parametrization of the text generation process
+    """
+
+    do_sample: Optional[bool]
+    """Whether to use sampling instead of greedy decoding when generating new tokens."""
+    early_stopping: Optional[Union[bool, "EarlyStoppingEnum"]]
+    """Controls the stopping condition for beam-based methods."""
+    epsilon_cutoff: Optional[float]
+    """If set to float strictly between 0 and 1, only tokens with a conditional probability
+    greater than epsilon_cutoff will be sampled. In the paper, suggested values range from
+    3e-4 to 9e-4, depending on the size of the model. See [Truncation Sampling as Language
+    Model Desmoothing](https://hf.co/papers/2210.15191) for more details.
+    """
+    eta_cutoff: Optional[float]
+    """Eta sampling is a hybrid of locally typical sampling and epsilon sampling. If set to
+    float strictly between 0 and 1, a token is only considered if it is greater than either
+    eta_cutoff or sqrt(eta_cutoff) * exp(-entropy(softmax(next_token_logits))). The latter
+    term is intuitively the expected next token probability, scaled by sqrt(eta_cutoff). In
+    the paper, suggested values range from 3e-4 to 2e-3, depending on the size of the model.
+    See [Truncation Sampling as Language Model Desmoothing](https://hf.co/papers/2210.15191)
+    for more details.
+    """
+    max_length: Optional[int]
+    """The maximum length (in tokens) of the generated text, including the input."""
+    max_new_tokens: Optional[int]
+    """The maximum number of tokens to generate. Takes precedence over maxLength."""
+    min_length: Optional[int]
+    """The minimum length (in tokens) of the generated text, including the input."""
+    min_new_tokens: Optional[int]
+    """The minimum number of tokens to generate. Takes precedence over maxLength."""
+    num_beam_groups: Optional[int]
+    """Number of groups to divide num_beams into in order to ensure diversity among different
+    groups of beams. See [this paper](https://hf.co/papers/1610.02424) for more details.
+    """
+    num_beams: Optional[int]
+    """Number of beams to use for beam search."""
+    penalty_alpha: Optional[float]
+    """The value balances the model confidence and the degeneration penalty in contrastive
+    search decoding.
+    """
+    temperature: Optional[float]
+    """The value used to modulate the next token probabilities."""
+    top_k: Optional[int]
+    """The number of highest probability vocabulary tokens to keep for top-k-filtering."""
+    top_p: Optional[float]
+    """If set to float < 1, only the smallest set of most probable tokens with probabilities
+    that add up to top_p or higher are kept for generation.
+    """
+    typical_p: Optional[float]
+    """Local typicality measures how similar the conditional probability of predicting a target
+    token next is to the expected conditional probability of predicting a random token next,
+    given the partial text already generated. If set to float < 1, the smallest set of the
+    most locally typical tokens with probabilities that add up to typical_p or higher are
+    kept for generation. See [this paper](https://hf.co/papers/2202.00666) for more details.
+    """
+    use_cache: Optional[bool]
+    """Whether the model should use the past last key/values attentions to speed up decoding"""
+
+
+@dataclass
+class TextToAudioParameters(BaseInferenceType):
+    """Additional inference parameters
+    Additional inference parameters for Text To Audio
+    """
+
+    generate: Optional[GenerationParameters]
+    """Parametrization of the text generation process"""
+
+
+@dataclass
+class TextToSpeechInput(BaseInferenceType):
+    """Inputs for Text to Speech inference
+    Inputs for Text To Audio inference
+    """
+
+    inputs: str
+    """The input text data"""
+    parameters: Optional[TextToAudioParameters]
+    """Additional inference parameters"""
+
+
+@dataclass
+class TextToSpeechOutput(BaseInferenceType):
+    """Outputs for Text to Speech inference
+    Outputs of inference for the Text To Audio task
+    """
+
+    audio: Any
+    """The generated audio waveform."""
+    sampling_rate: Any
+    text_to_speech_output_sampling_rate: Optional[float]
+    """The sampling rate of the generated audio waveform."""
diff --git a/src/huggingface_hub/inference/_generated/types/token_classification.py b/src/huggingface_hub/inference/_generated/types/token_classification.py
new file mode 100644
index 00000000..c036c5fc
--- /dev/null
+++ b/src/huggingface_hub/inference/_generated/types/token_classification.py
@@ -0,0 +1,53 @@
+# Inference code generated from the JSON schema spec in @huggingface/tasks.
+#
+# See:
+#   - script: https://github.com/huggingface/huggingface.js/blob/main/packages/tasks/scripts/inference-codegen.ts
+#   - specs:  https://github.com/huggingface/huggingface.js/tree/main/packages/tasks/src/tasks.
+from dataclasses import dataclass
+from typing import Any, List, Literal, Optional
+
+from .base import BaseInferenceType
+
+
+TokenClassificationAggregationStrategy = Literal["none", "simple", "first", "average", "max"]
+
+
+@dataclass
+class TokenClassificationParameters(BaseInferenceType):
+    """Additional inference parameters
+    Additional inference parameters for Token Classification
+    """
+
+    aggregation_strategy: Optional["TokenClassificationAggregationStrategy"]
+    """The strategy used to fuse tokens based on model predictions"""
+    ignore_labels: Optional[List[str]]
+    """A list of labels to ignore"""
+    stride: Optional[int]
+    """The number of overlapping tokens between chunks when splitting the input text."""
+
+
+@dataclass
+class TokenClassificationInput(BaseInferenceType):
+    """Inputs for Token Classification inference"""
+
+    inputs: str
+    """The input text data"""
+    parameters: Optional[TokenClassificationParameters]
+    """Additional inference parameters"""
+
+
+@dataclass
+class TokenClassificationOutputElement(BaseInferenceType):
+    """Outputs of inference for the Token Classification task"""
+
+    label: Any
+    score: float
+    """The associated score / probability"""
+    end: Optional[int]
+    """The character position in the input where this group ends."""
+    entity_group: Optional[str]
+    """The predicted label for that group of tokens"""
+    start: Optional[int]
+    """The character position in the input where this group begins."""
+    word: Optional[str]
+    """The corresponding text"""
diff --git a/src/huggingface_hub/inference/_generated/types/translation.py b/src/huggingface_hub/inference/_generated/types/translation.py
new file mode 100644
index 00000000..9ef4df16
--- /dev/null
+++ b/src/huggingface_hub/inference/_generated/types/translation.py
@@ -0,0 +1,49 @@
+# Inference code generated from the JSON schema spec in @huggingface/tasks.
+#
+# See:
+#   - script: https://github.com/huggingface/huggingface.js/blob/main/packages/tasks/scripts/inference-codegen.ts
+#   - specs:  https://github.com/huggingface/huggingface.js/tree/main/packages/tasks/src/tasks.
+from dataclasses import dataclass
+from typing import Any, Dict, Literal, Optional
+
+from .base import BaseInferenceType
+
+
+Text2TextGenerationTruncationStrategy = Literal["do_not_truncate", "longest_first", "only_first", "only_second"]
+
+
+@dataclass
+class Text2TextGenerationParameters(BaseInferenceType):
+    """Additional inference parameters
+    Additional inference parameters for Text2text Generation
+    """
+
+    clean_up_tokenization_spaces: Optional[bool]
+    """Whether to clean up the potential extra spaces in the text output."""
+    generate_parameters: Optional[Dict[str, Any]]
+    """Additional parametrization of the text generation algorithm"""
+    truncation: Optional["Text2TextGenerationTruncationStrategy"]
+    """The truncation strategy to use"""
+
+
+@dataclass
+class TranslationInput(BaseInferenceType):
+    """Inputs for Translation inference
+    Inputs for Text2text Generation inference
+    """
+
+    inputs: str
+    """The input text data"""
+    parameters: Optional[Text2TextGenerationParameters]
+    """Additional inference parameters"""
+
+
+@dataclass
+class TranslationOutput(BaseInferenceType):
+    """Outputs for Translation inference
+    Outputs of inference for the Text2text Generation task
+    """
+
+    translation_text: Any
+    translation_output_translation_text: Optional[str]
+    """The translated text."""
diff --git a/src/huggingface_hub/inference/_generated/types/video_classification.py b/src/huggingface_hub/inference/_generated/types/video_classification.py
new file mode 100644
index 00000000..9e4d6eeb
--- /dev/null
+++ b/src/huggingface_hub/inference/_generated/types/video_classification.py
@@ -0,0 +1,47 @@
+# Inference code generated from the JSON schema spec in @huggingface/tasks.
+#
+# See:
+#   - script: https://github.com/huggingface/huggingface.js/blob/main/packages/tasks/scripts/inference-codegen.ts
+#   - specs:  https://github.com/huggingface/huggingface.js/tree/main/packages/tasks/src/tasks.
+from dataclasses import dataclass
+from typing import Any, Literal, Optional
+
+from .base import BaseInferenceType
+
+
+ClassificationOutputTransform = Literal["sigmoid", "softmax", "none"]
+
+
+@dataclass
+class VideoClassificationParameters(BaseInferenceType):
+    """Additional inference parameters
+    Additional inference parameters for Video Classification
+    """
+
+    frame_sampling_rate: Optional[int]
+    """The sampling rate used to select frames from the video."""
+    function_to_apply: Optional["ClassificationOutputTransform"]
+    num_frames: Optional[int]
+    """The number of sampled frames to consider for classification."""
+    top_k: Optional[int]
+    """When specified, limits the output to the top K most probable classes."""
+
+
+@dataclass
+class VideoClassificationInput(BaseInferenceType):
+    """Inputs for Video Classification inference"""
+
+    inputs: Any
+    """The input video data"""
+    parameters: Optional[VideoClassificationParameters]
+    """Additional inference parameters"""
+
+
+@dataclass
+class VideoClassificationOutputElement(BaseInferenceType):
+    """Outputs of inference for the Video Classification task"""
+
+    label: str
+    """The predicted class label."""
+    score: float
+    """The corresponding probability."""
diff --git a/src/huggingface_hub/inference/_generated/types/visual_question_answering.py b/src/huggingface_hub/inference/_generated/types/visual_question_answering.py
new file mode 100644
index 00000000..bc91d49f
--- /dev/null
+++ b/src/huggingface_hub/inference/_generated/types/visual_question_answering.py
@@ -0,0 +1,53 @@
+# Inference code generated from the JSON schema spec in @huggingface/tasks.
+#
+# See:
+#   - script: https://github.com/huggingface/huggingface.js/blob/main/packages/tasks/scripts/inference-codegen.ts
+#   - specs:  https://github.com/huggingface/huggingface.js/tree/main/packages/tasks/src/tasks.
+from dataclasses import dataclass
+from typing import Any, Optional
+
+from .base import BaseInferenceType
+
+
+@dataclass
+class VisualQuestionAnsweringInputData(BaseInferenceType):
+    """One (image, question) pair to answer"""
+
+    image: Any
+    """The image."""
+    question: Any
+    """The question to answer based on the image."""
+
+
+@dataclass
+class VisualQuestionAnsweringParameters(BaseInferenceType):
+    """Additional inference parameters
+    Additional inference parameters for Visual Question Answering
+    """
+
+    top_k: Optional[int]
+    """The number of answers to return (will be chosen by order of likelihood). Note that we
+    return less than topk answers if there are not enough options available within the
+    context.
+    """
+
+
+@dataclass
+class VisualQuestionAnsweringInput(BaseInferenceType):
+    """Inputs for Visual Question Answering inference"""
+
+    inputs: VisualQuestionAnsweringInputData
+    """One (image, question) pair to answer"""
+    parameters: Optional[VisualQuestionAnsweringParameters]
+    """Additional inference parameters"""
+
+
+@dataclass
+class VisualQuestionAnsweringOutputElement(BaseInferenceType):
+    """Outputs of inference for the Visual Question Answering task"""
+
+    label: Any
+    score: float
+    """The associated score / probability"""
+    answer: Optional[str]
+    """The answer to the question"""
diff --git a/src/huggingface_hub/inference/_generated/types/zero_shot_classification.py b/src/huggingface_hub/inference/_generated/types/zero_shot_classification.py
new file mode 100644
index 00000000..1570da22
--- /dev/null
+++ b/src/huggingface_hub/inference/_generated/types/zero_shot_classification.py
@@ -0,0 +1,56 @@
+# Inference code generated from the JSON schema spec in @huggingface/tasks.
+#
+# See:
+#   - script: https://github.com/huggingface/huggingface.js/blob/main/packages/tasks/scripts/inference-codegen.ts
+#   - specs:  https://github.com/huggingface/huggingface.js/tree/main/packages/tasks/src/tasks.
+from dataclasses import dataclass
+from typing import List, Optional
+
+from .base import BaseInferenceType
+
+
+@dataclass
+class ZeroShotClassificationInputData(BaseInferenceType):
+    """The input text data, with candidate labels"""
+
+    candidate_labels: List[str]
+    """The set of possible class labels to classify the text into."""
+    text: str
+    """The text to classify"""
+
+
+@dataclass
+class ZeroShotClassificationParameters(BaseInferenceType):
+    """Additional inference parameters
+    Additional inference parameters for Zero Shot Classification
+    """
+
+    hypothesis_template: Optional[str]
+    """The sentence used in conjunction with candidateLabels to attempt the text classification
+    by replacing the placeholder with the candidate labels.
+    """
+    multi_label: Optional[bool]
+    """Whether multiple candidate labels can be true. If false, the scores are normalized such
+    that the sum of the label likelihoods for each sequence is 1. If true, the labels are
+    considered independent and probabilities are normalized for each candidate.
+    """
+
+
+@dataclass
+class ZeroShotClassificationInput(BaseInferenceType):
+    """Inputs for Zero Shot Classification inference"""
+
+    inputs: ZeroShotClassificationInputData
+    """The input text data, with candidate labels"""
+    parameters: Optional[ZeroShotClassificationParameters]
+    """Additional inference parameters"""
+
+
+@dataclass
+class ZeroShotClassificationOutputElement(BaseInferenceType):
+    """Outputs of inference for the Zero Shot Classification task"""
+
+    label: str
+    """The predicted class label."""
+    score: float
+    """The corresponding probability."""
diff --git a/src/huggingface_hub/inference/_generated/types/zero_shot_image_classification.py b/src/huggingface_hub/inference/_generated/types/zero_shot_image_classification.py
new file mode 100644
index 00000000..172dca9a
--- /dev/null
+++ b/src/huggingface_hub/inference/_generated/types/zero_shot_image_classification.py
@@ -0,0 +1,51 @@
+# Inference code generated from the JSON schema spec in @huggingface/tasks.
+#
+# See:
+#   - script: https://github.com/huggingface/huggingface.js/blob/main/packages/tasks/scripts/inference-codegen.ts
+#   - specs:  https://github.com/huggingface/huggingface.js/tree/main/packages/tasks/src/tasks.
+from dataclasses import dataclass
+from typing import Any, List, Optional
+
+from .base import BaseInferenceType
+
+
+@dataclass
+class ZeroShotImageClassificationInputData(BaseInferenceType):
+    """The input image data, with candidate labels"""
+
+    candidate_labels: List[str]
+    """The candidate labels for this image"""
+    image: Any
+    """The image data to classify"""
+
+
+@dataclass
+class ZeroShotImageClassificationParameters(BaseInferenceType):
+    """Additional inference parameters
+    Additional inference parameters for Zero Shot Image Classification
+    """
+
+    hypothesis_template: Optional[str]
+    """The sentence used in conjunction with candidateLabels to attempt the text classification
+    by replacing the placeholder with the candidate labels.
+    """
+
+
+@dataclass
+class ZeroShotImageClassificationInput(BaseInferenceType):
+    """Inputs for Zero Shot Image Classification inference"""
+
+    inputs: ZeroShotImageClassificationInputData
+    """The input image data, with candidate labels"""
+    parameters: Optional[ZeroShotImageClassificationParameters]
+    """Additional inference parameters"""
+
+
+@dataclass
+class ZeroShotImageClassificationOutputElement(BaseInferenceType):
+    """Outputs of inference for the Zero Shot Image Classification task"""
+
+    label: str
+    """The predicted class label."""
+    score: float
+    """The corresponding probability."""
diff --git a/src/huggingface_hub/inference/_generated/types/zero_shot_object_detection.py b/src/huggingface_hub/inference/_generated/types/zero_shot_object_detection.py
new file mode 100644
index 00000000..467a6f36
--- /dev/null
+++ b/src/huggingface_hub/inference/_generated/types/zero_shot_object_detection.py
@@ -0,0 +1,55 @@
+# Inference code generated from the JSON schema spec in @huggingface/tasks.
+#
+# See:
+#   - script: https://github.com/huggingface/huggingface.js/blob/main/packages/tasks/scripts/inference-codegen.ts
+#   - specs:  https://github.com/huggingface/huggingface.js/tree/main/packages/tasks/src/tasks.
+from dataclasses import dataclass
+from typing import Any, Dict, List, Optional
+
+from .base import BaseInferenceType
+
+
+@dataclass
+class ZeroShotObjectDetectionInputData(BaseInferenceType):
+    """The input image data, with candidate labels"""
+
+    candidate_labels: List[str]
+    """The candidate labels for this image"""
+    image: Any
+    """The image data to generate bounding boxes from"""
+
+
+@dataclass
+class ZeroShotObjectDetectionInput(BaseInferenceType):
+    """Inputs for Zero Shot Object Detection inference"""
+
+    inputs: ZeroShotObjectDetectionInputData
+    """The input image data, with candidate labels"""
+    parameters: Optional[Dict[str, Any]]
+    """Additional inference parameters"""
+
+
+@dataclass
+class BoundingBox(BaseInferenceType):
+    """The predicted bounding box. Coordinates are relative to the top left corner of the input
+    image.
+    """
+
+    xmax: int
+    xmin: int
+    ymax: int
+    ymin: int
+
+
+@dataclass
+class ZeroShotObjectDetectionOutputElement(BaseInferenceType):
+    """Outputs of inference for the Zero Shot Object Detection task"""
+
+    box: BoundingBox
+    """The predicted bounding box. Coordinates are relative to the top left corner of the input
+    image.
+    """
+    label: str
+    """A candidate label"""
+    score: float
+    """The associated score / probability"""
diff --git a/src/huggingface_hub/inference/_types.py b/src/huggingface_hub/inference/_types.py
index 3d15fc3f..70c21372 100644
--- a/src/huggingface_hub/inference/_types.py
+++ b/src/huggingface_hub/inference/_types.py
@@ -12,42 +12,13 @@
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
-from typing import TYPE_CHECKING, List, TypedDict
 
+from typing import List, TypedDict
 
-if TYPE_CHECKING:
-    from PIL import Image
 
-
-class AudioToAudioOutput(TypedDict):
-    """Dictionary containing the output of a [`~InferenceClient.audio_to_audio`] task.
-
-    Args:
-        label (`str`):
-            The label of the audio file.
-        content-type (`str`):
-            The content type of audio file.
-        blob (`bytes`):
-            The audio file in byte format.
-    """
-
-    label: str
-    content_type: str
-    blob: bytes
-
-
-class ClassificationOutput(TypedDict):
-    """Dictionary containing the output of a [`~InferenceClient.audio_classification`] and  [`~InferenceClient.image_classification`] task.
-
-    Args:
-        label (`str`):
-            The label predicted by the model.
-        score (`float`):
-            The score of the label predicted by the model.
-    """
-
-    label: str
-    score: float
+# Legacy types
+# Types are now generated from the JSON schema spec in @huggingface/tasks.
+# See ./src/huggingface_hub/inference/_generated/types
 
 
 class ConversationalOutputConversation(TypedDict):
@@ -79,122 +50,3 @@ class ConversationalOutput(TypedDict):
     conversation: ConversationalOutputConversation
     generated_text: str
     warnings: List[str]
-
-
-class FillMaskOutput(TypedDict):
-    """Dictionary containing information about a [`~InferenceClient.fill_mask`] task.
-
-    Args:
-        score (`float`):
-            The probability of the token.
-        token (`int`):
-            The id of the token.
-        token_str (`str`):
-            The string representation of the token.
-        sequence (`str`):
-            The actual sequence of tokens that ran against the model (may contain special tokens).
-    """
-
-    score: float
-    token: int
-    token_str: str
-    sequence: str
-
-
-class ImageSegmentationOutput(TypedDict):
-    """Dictionary containing information about a [`~InferenceClient.image_segmentation`] task. In practice, image segmentation returns a
-    list of `ImageSegmentationOutput` with 1 item per mask.
-
-    Args:
-        label (`str`):
-            The label corresponding to the mask.
-        mask (`Image`):
-            An Image object representing the mask predicted by the model.
-        score (`float`):
-            The score associated with the label for this mask.
-    """
-
-    label: str
-    mask: "Image"
-    score: float
-
-
-class ObjectDetectionOutput(TypedDict):
-    """Dictionary containing information about a [`~InferenceClient.object_detection`] task.
-
-    Args:
-        label (`str`):
-            The label corresponding to the detected object.
-        box (`dict`):
-            A dict response of bounding box coordinates of
-            the detected object: xmin, ymin, xmax, ymax
-        score (`float`):
-            The score corresponding to the detected object.
-    """
-
-    label: str
-    box: dict
-    score: float
-
-
-class QuestionAnsweringOutput(TypedDict):
-    """Dictionary containing information about a [`~InferenceClient.question_answering`] task.
-
-    Args:
-        score (`float`):
-            A float that represents how likely that the answer is correct.
-        start (`int`):
-            The index (string wise) of the start of the answer within context.
-        end (`int`):
-            The index (string wise) of the end of the answer within context.
-        answer (`str`):
-            A string that is the answer within the text.
-    """
-
-    score: float
-    start: int
-    end: int
-    answer: str
-
-
-class TableQuestionAnsweringOutput(TypedDict):
-    """Dictionary containing information about a [`~InferenceClient.table_question_answering`] task.
-
-    Args:
-        answer (`str`):
-            The plaintext answer.
-        coordinates (`List[List[int]]`):
-            A list of coordinates of the cells referenced in the answer.
-        cells (`List[int]`):
-            A list of coordinates of the cells contents.
-        aggregator (`str`):
-            The aggregator used to get the answer.
-    """
-
-    answer: str
-    coordinates: List[List[int]]
-    cells: List[List[int]]
-    aggregator: str
-
-
-class TokenClassificationOutput(TypedDict):
-    """Dictionary containing the output of a [`~InferenceClient.token_classification`] task.
-
-    Args:
-        entity_group (`str`):
-            The type for the entity being recognized (model specific).
-        score (`float`):
-            The score of the label predicted by the model.
-        word (`str`):
-            The string that was captured.
-        start (`int`):
-            The offset stringwise where the answer is located. Useful to disambiguate if word occurs multiple times.
-        end (`int`):
-            The offset stringwise where the answer is located. Useful to disambiguate if word occurs multiple times.
-    """
-
-    entity_group: str
-    score: float
-    word: str
-    start: int
-    end: int
diff --git a/utils/generate_inference_types.py b/utils/generate_inference_types.py
new file mode 100644
index 00000000..9c9a782e
--- /dev/null
+++ b/utils/generate_inference_types.py
@@ -0,0 +1,235 @@
+# coding=utf-8
+# Copyright 2024-present, the HuggingFace Inc. team.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""Contains a tool to generate `src/huggingface_hub/inference/_generated/types`."""
+import argparse
+import os
+import re
+import tempfile
+from pathlib import Path
+from typing import Dict, List, NoReturn
+
+from helpers import check_and_update_file_content
+from ruff.__main__ import find_ruff_bin
+
+
+huggingface_hub_folder_path = Path(__file__).parents[1] / "src" / "huggingface_hub"
+INFERENCE_TYPES_FOLDER_PATH = huggingface_hub_folder_path / "inference" / "_generated" / "types"
+MAIN_INIT_PY_FILE = huggingface_hub_folder_path / "__init__.py"
+REFERENCE_PACKAGE_PATH = (
+    Path(__file__).parents[1] / "docs" / "source" / "en" / "package_reference" / "inference_types.md"
+)
+
+IGNORE_FILES = [
+    "__init__.py",
+    "base.py",
+]
+
+BASE_DATACLASS_REGEX = re.compile(
+    r"""
+    ^@dataclass
+    \nclass\s(\w+):\n
+""",
+    re.VERBOSE | re.MULTILINE,
+)
+
+INHERITED_DATACLASS_REGEX = re.compile(
+    r"""
+    ^@dataclass
+    \nclass\s(\w+)\(BaseInferenceType\):
+""",
+    re.VERBOSE | re.MULTILINE,
+)
+
+OPTIONAL_FIELD_REGEX = re.compile(r": Optional\[(.+)\]$", re.MULTILINE)
+
+
+INIT_PY_HEADER = """
+# This file is auto-generated by `utils/generate_inference_types.py`.
+# Do not modify it manually.
+#
+# ruff: noqa: F401
+
+from .base import BaseInferenceType
+"""
+
+# Regex to add all dataclasses to ./src/huggingface_hub/__init__.py
+MAIN_INIT_PY_REGEX = re.compile(
+    r"""
+\"inference\._generated\.types\":\s*\[ # module name
+    (.*?) # all dataclasses listed
+\] # closing bracket
+""",
+    re.MULTILINE | re.VERBOSE | re.DOTALL,
+)
+
+# List of classes that are shared across multiple modules
+# This is used to fix the naming of the classes (to make them unique by task)
+SHARED_CLASSES = [
+    "ClassificationOutputTransform",
+    "ClassificationOutput",
+]
+
+REFERENCE_PACKAGE_CONTENT = """
+<!--⚠️ Note that this file is in Markdown but contain specific syntax for our doc-builder (similar to MDX) that may not be
+rendered properly in your Markdown viewer.
+-->
+<!--⚠️ Note that this file is auto-generated by `utils/generate_inference_types.py`. Do not modify it manually.-->
+
+
+# Inference types
+
+This page lists the types (e.g. dataclasses) available for each task supported on the Hugging Face Hub.
+Each task is specified using a JSON schema, and the types are generated from these schemas - with some customization
+due to Python requirements.
+Visit [@huggingface.js/tasks](https://github.com/huggingface/huggingface.js/tree/main/packages/tasks/src/tasks)
+to find the JSON schemas for each task.
+
+This part of the lib is still under development and will be improved in future releases.
+
+
+{types}
+"""
+
+
+def _inherit_from_base(content: str) -> str:
+    content = content.replace(
+        "\nfrom dataclasses import", "\nfrom .base import BaseInferenceType\nfrom dataclasses import"
+    )
+    content = BASE_DATACLASS_REGEX.sub(r"@dataclass\nclass \1(BaseInferenceType):\n", content)
+    return content
+
+
+def _delete_empty_lines(content: str) -> str:
+    return "\n".join([line for line in content.split("\n") if line.strip()])
+
+
+def _fix_naming_for_shared_classes(content: str, module_name: str) -> str:
+    for cls in SHARED_CLASSES:
+        cls_definition = f"\nclass {cls}"
+        if cls_definition in content:
+            # Very hacky way to build "AudioClassificationOutputElement" instead of "ClassificationOutput"
+            new_cls_definition = "\nclass " + "".join(part.capitalize() for part in module_name.split("_"))
+            if "Classification" in new_cls_definition:
+                # to avoid "ClassificationClassificationOutput"
+                new_cls_definition += cls.removeprefix("Classification")
+            else:
+                new_cls_definition += cls
+            if new_cls_definition.endswith("ClassificationOutput"):
+                # to get "AudioClassificationOutputElement"
+                new_cls_definition += "Element"
+            content = content.replace(cls_definition, new_cls_definition)
+    return content
+
+
+def _list_dataclasses(content: str) -> List[str]:
+    """List all dataclasses defined in the module."""
+    return INHERITED_DATACLASS_REGEX.findall(content)
+
+
+def fix_inference_classes(content: str, module_name: str) -> str:
+    content = _inherit_from_base(content)
+    content = _delete_empty_lines(content)
+    content = _fix_naming_for_shared_classes(content, module_name)
+    return content
+
+
+def create_init_py(dataclasses: Dict[str, List[str]]):
+    """Create __init__.py file with all dataclasses."""
+    content = INIT_PY_HEADER
+    content += "\n"
+    content += "\n".join(
+        [f"from .{module} import {', '.join(dataclasses_list)}" for module, dataclasses_list in dataclasses.items()]
+    )
+    return content
+
+
+def add_dataclasses_to_main_init(content: str, dataclasses: Dict[str, List[str]]):
+    dataclasses_list = sorted({cls for classes in dataclasses.values() for cls in classes})
+    dataclasses_str = ", ".join(f"'{cls}'" for cls in dataclasses_list)
+
+    return MAIN_INIT_PY_REGEX.sub(f'"inference._generated.types": [{dataclasses_str}]', content)
+
+
+def format_source_code(code: str) -> str:
+    """Apply formatter on the generated source code."""
+    with tempfile.TemporaryDirectory() as tmpdir:
+        filepath = Path(tmpdir) / "tmp.py"
+        filepath.write_text(code)
+        ruff_bin = find_ruff_bin()
+        os.spawnv(os.P_WAIT, ruff_bin, ["ruff", str(filepath), "--fix", "--quiet"])
+        os.spawnv(os.P_WAIT, ruff_bin, ["ruff", "format", str(filepath), "--quiet"])
+        return filepath.read_text()
+
+
+def generate_reference_package(dataclasses: Dict[str, List[str]]) -> str:
+    """Generate the reference package content."""
+
+    per_task_docs = []
+    for task in sorted(dataclasses.keys()):
+        lines = [f"[[autodoc]] huggingface_hub.{cls}" for cls in sorted(dataclasses[task])]
+        lines_str = "\n\n".join(lines)
+        per_task_docs.append(f"\n## {task}\n\n{lines_str}\n\n")
+
+    return REFERENCE_PACKAGE_CONTENT.format(types="\n".join(per_task_docs))
+
+
+def check_inference_types(update: bool) -> NoReturn:
+    """Check AsyncInferenceClient is correctly defined and consistent with InferenceClient.
+
+    This script is used in the `make style` and `make quality` checks.
+    """
+    dataclasses = {}
+    for file in INFERENCE_TYPES_FOLDER_PATH.glob("*.py"):
+        if file.name in IGNORE_FILES:
+            continue
+
+        content = file.read_text()
+
+        fixed_content = fix_inference_classes(content, module_name=file.stem)
+        formatted_content = format_source_code(fixed_content)
+
+        dataclasses[file.stem] = _list_dataclasses(formatted_content)
+
+        check_and_update_file_content(file, formatted_content, update)
+
+    init_py_content = create_init_py(dataclasses)
+    init_py_content = format_source_code(init_py_content)
+    init_py_file = INFERENCE_TYPES_FOLDER_PATH / "__init__.py"
+    check_and_update_file_content(init_py_file, init_py_content, update)
+
+    main_init_py_content = MAIN_INIT_PY_FILE.read_text()
+    updated_main_init_py_content = add_dataclasses_to_main_init(main_init_py_content, dataclasses)
+    updated_main_init_py_content = format_source_code(updated_main_init_py_content)
+    check_and_update_file_content(MAIN_INIT_PY_FILE, updated_main_init_py_content, update)
+
+    reference_package_content = generate_reference_package(dataclasses)
+    check_and_update_file_content(REFERENCE_PACKAGE_PATH, reference_package_content, update)
+
+    print("✅ All good! (inference types)")
+    exit(0)
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser()
+    parser.add_argument(
+        "--update",
+        action="store_true",
+        help=(
+            "Whether to re-generate files in `./src/huggingface_hub/inference/_generated/types/` if a change is detected."
+        ),
+    )
+    args = parser.parse_args()
+
+    check_inference_types(update=args.update)
diff --git a/utils/helpers.py b/utils/helpers.py
new file mode 100644
index 00000000..ac38c259
--- /dev/null
+++ b/utils/helpers.py
@@ -0,0 +1,27 @@
+# coding=utf-8
+# Copyright 2024-present, the HuggingFace Inc. team.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""Contains helpers used by the scripts in `./utils`."""
+from pathlib import Path
+
+
+def check_and_update_file_content(file: Path, expected_content: str, update: bool):
+    content = file.read_text() if file.exists() else None
+    if content != expected_content:
+        if update:
+            file.write_text(expected_content)
+            print(f"  {file} has been updated. Please make sure the changes are accurate and commit them.")
+        else:
+            print(f"❌ Expected content mismatch in {file}.")
+            exit(1)


Test patch:
diff --git a/tests/test_inference_client.py b/tests/test_inference_client.py
index c7e7c441..b3904c6c 100644
--- a/tests/test_inference_client.py
+++ b/tests/test_inference_client.py
@@ -20,7 +20,23 @@ import numpy as np
 import pytest
 from PIL import Image
 
-from huggingface_hub import InferenceClient, hf_hub_download
+from huggingface_hub import (
+    AutomaticSpeechRecognitionOutput,
+    DocumentQuestionAnsweringOutputElement,
+    FillMaskOutputElement,
+    ImageClassificationOutputElement,
+    InferenceClient,
+    ObjectDetectionOutputElement,
+    QuestionAnsweringOutputElement,
+    SummarizationOutput,
+    TableQuestionAnsweringOutputElement,
+    TextClassificationOutputElement,
+    TokenClassificationOutputElement,
+    TranslationOutput,
+    VisualQuestionAnsweringOutputElement,
+    ZeroShotClassificationOutputElement,
+    hf_hub_download,
+)
 from huggingface_hub.constants import ALL_INFERENCE_API_FRAMEWORKS, MAIN_INFERENCE_API_FRAMEWORKS
 from huggingface_hub.inference._client import _open_as_binary
 from huggingface_hub.utils import HfHubHTTPError, build_hf_headers
@@ -92,21 +108,24 @@ class InferenceClientVCRTest(InferenceClientTest):
         self.assertIsInstance(output, list)
         self.assertGreater(len(output), 0)
         for item in output:
-            self.assertIsInstance(item["score"], float)
-            self.assertIsInstance(item["label"], str)
+            self.assertIsInstance(item.score, float)
+            self.assertIsInstance(item.label, str)
 
     def test_audio_to_audio(self) -> None:
         output = self.client.audio_to_audio(self.audio_file)
         assert isinstance(output, list)
         assert len(output) > 0
         for item in output:
-            assert isinstance(item["label"], str)
-            assert isinstance(item["blob"], bytes)
-            assert item["content-type"] == "audio/flac"
+            assert isinstance(item.label, str)
+            assert isinstance(item.blob, bytes)
+            assert item.content_type == "audio/flac"
 
     def test_automatic_speech_recognition(self) -> None:
         output = self.client.automatic_speech_recognition(self.audio_file)
-        self.assertEqual(output, "A MAN SAID TO THE UNIVERSE SIR I EXIST")
+        assert output == AutomaticSpeechRecognitionOutput(
+            text="A MAN SAID TO THE UNIVERSE SIR I EXIST",
+            chunks=None,
+        )
 
     def test_conversational(self) -> None:
         output = self.client.conversational("Hi, who are you?")
@@ -141,7 +160,14 @@ class InferenceClientVCRTest(InferenceClientTest):
 
     def test_document_question_answering(self) -> None:
         output = self.client.document_question_answering(self.document_file, "What is the purchase amount?")
-        self.assertEqual(output, [{"answer": "$1,000,000,000"}])
+        self.assertEqual(
+            output,
+            [
+                DocumentQuestionAnsweringOutputElement(
+                    answer="$1,000,000,000", end=None, score=None, start=None, words=None
+                )
+            ],
+        )
 
     def test_feature_extraction_with_transformers(self) -> None:
         embedding = self.client.feature_extraction("Hi, who are you?")
@@ -156,13 +182,43 @@ class InferenceClientVCRTest(InferenceClientTest):
     def test_fill_mask(self) -> None:
         model = "distilroberta-base"
         output = self.client.fill_mask("The goal of life is <mask>.", model=model)
-        self.assertIsInstance(output, list)
-        self.assertEqual(len(output[0]), 4)
-        self.assertIsInstance(output[0], dict)
-        self.assertEqual(
-            set(k for el in output for k in el.keys()),
-            {"score", "sequence", "token", "token_str"},
-        )
+        assert output == [
+            FillMaskOutputElement(
+                score=0.06897063553333282,
+                sequence="The goal of life is happiness.",
+                token=11098,
+                token_str=" happiness",
+                fill_mask_output_token_str=None,
+            ),
+            FillMaskOutputElement(
+                score=0.06554922461509705,
+                sequence="The goal of life is immortality.",
+                token=45075,
+                token_str=" immortality",
+                fill_mask_output_token_str=None,
+            ),
+            FillMaskOutputElement(
+                score=0.0323575921356678,
+                sequence="The goal of life is yours.",
+                token=14314,
+                token_str=" yours",
+                fill_mask_output_token_str=None,
+            ),
+            FillMaskOutputElement(
+                score=0.02431388944387436,
+                sequence="The goal of life is liberation.",
+                token=22211,
+                token_str=" liberation",
+                fill_mask_output_token_str=None,
+            ),
+            FillMaskOutputElement(
+                score=0.023767812177538872,
+                sequence="The goal of life is simplicity.",
+                token=25342,
+                token_str=" simplicity",
+                fill_mask_output_token_str=None,
+            ),
+        ]
 
     def test_get_recommended_model_has_recommendation(self) -> None:
         assert self.client.get_recommended_model("feature-extraction") == "facebook/bart-base"
@@ -174,22 +230,24 @@ class InferenceClientVCRTest(InferenceClientTest):
 
     def test_image_classification(self) -> None:
         output = self.client.image_classification(self.image_file)
-        self.assertIsInstance(output, list)
-        self.assertGreater(len(output), 0)
-        for item in output:
-            self.assertIsInstance(item["score"], float)
-            self.assertIsInstance(item["label"], str)
+        assert output == [
+            ImageClassificationOutputElement(label="brassiere, bra, bandeau", score=0.1176738440990448),
+            ImageClassificationOutputElement(label="sombrero", score=0.0957278460264206),
+            ImageClassificationOutputElement(label="cowboy hat, ten-gallon hat", score=0.09000881016254425),
+            ImageClassificationOutputElement(label="bonnet, poke bonnet", score=0.06615243852138519),
+            ImageClassificationOutputElement(label="fur coat", score=0.06151164695620537),
+        ]
 
     def test_image_segmentation(self) -> None:
         output = self.client.image_segmentation(self.image_file)
-        self.assertIsInstance(output, list)
-        self.assertGreater(len(output), 0)
+        assert isinstance(output, list)
+        assert len(output) > 0
         for item in output:
-            self.assertIsInstance(item["score"], float)
-            self.assertIsInstance(item["label"], str)
-            self.assertIsInstance(item["mask"], Image.Image)
-            self.assertEqual(item["mask"].height, 512)
-            self.assertEqual(item["mask"].width, 512)
+            assert isinstance(item.score, float)
+            assert isinstance(item.label, str)
+            assert isinstance(item.mask, Image.Image)
+            assert item.mask.height == 512
+            assert item.mask.width == 512
 
     # ERROR 500 from server
     # Only during tests, not when running locally. Has to be investigated.
@@ -207,26 +265,16 @@ class InferenceClientVCRTest(InferenceClientTest):
 
     def test_object_detection(self) -> None:
         output = self.client.object_detection(self.image_file)
-        self.assertIsInstance(output, list)
-        self.assertGreater(len(output), 0)
-        for item in output:
-            self.assertIsInstance(item["score"], float)
-            self.assertIsInstance(item["label"], str)
-            self.assertIsInstance(item["box"], dict)
-            self.assertIn("xmin", item["box"])
-            self.assertIn("ymin", item["box"])
-            self.assertIn("xmax", item["box"])
-            self.assertIn("ymax", item["box"])
+        assert output == [
+            ObjectDetectionOutputElement(
+                box={"xmin": 59, "ymin": 39, "xmax": 420, "ymax": 510}, label="person", score=0.9486683011054993
+            )
+        ]
 
     def test_question_answering(self) -> None:
         model = "deepset/roberta-base-squad2"
         output = self.client.question_answering(question="What is the meaning of life?", context="42", model=model)
-        self.assertIsInstance(output, dict)
-        self.assertGreater(len(output), 0)
-        self.assertIsInstance(output["score"], float)
-        self.assertIsInstance(output["start"], int)
-        self.assertIsInstance(output["end"], int)
-        self.assertEqual(output["answer"], "42")
+        assert output == QuestionAnsweringOutputElement(answer="42", end=2, score=1.4291124728060822e-08, start=0)
 
     def test_sentence_similarity(self) -> None:
         scores = self.client.sentence_similarity(
@@ -250,11 +298,12 @@ class InferenceClientVCRTest(InferenceClientTest):
             " metres (17 ft). Excluding transmitters, the Eiffel Tower is the second tallest free-standing structure"
             " in France after the Millau Viaduct."
         )
-        self.assertEqual(
-            summary,
-            "The tower is 324 metres (1,063 ft) tall, about the same height as an 81-storey building. Its base is"
-            " square, measuring 125 metres (410 ft) on each side. During its construction, the Eiffel Tower"
-            " surpassed the Washington Monument to become the tallest man-made structure in the world.",
+        assert summary == SummarizationOutput.parse_obj(
+            {
+                "summary_text": "The tower is 324 metres (1,063 ft) tall, about the same height as an 81-storey building. Its base is"
+                " square, measuring 125 metres (410 ft) on each side. During its construction, the Eiffel Tower"
+                " surpassed the Washington Monument to become the tallest man-made structure in the world.",
+            }
         )
 
     @pytest.mark.skip(reason="This model is not available on InferenceAPI")
@@ -295,20 +344,16 @@ class InferenceClientVCRTest(InferenceClientTest):
         }
         query = "How many stars does the transformers repository have?"
         output = self.client.table_question_answering(query=query, table=table)
-        self.assertEqual(type(output), dict)
-        self.assertEqual(len(output), 4)
-        self.assertEqual(
-            set(output.keys()),
-            {"aggregator", "answer", "cells", "coordinates"},
+        assert output == TableQuestionAnsweringOutputElement(
+            answer="AVERAGE > 36542", cells=["36542"], coordinates=[[0, 1]], aggregator="AVERAGE"
         )
 
     def test_text_classification(self) -> None:
         output = self.client.text_classification("I like you")
-        self.assertIsInstance(output, list)
-        self.assertEqual(len(output), 2)
-        for item in output:
-            self.assertIsInstance(item["score"], float)
-            self.assertIsInstance(item["label"], str)
+        assert output == [
+            TextClassificationOutputElement(label="POSITIVE", score=0.9998695850372314),
+            TextClassificationOutputElement(label="NEGATIVE", score=0.0001304351753788069),
+        ]
 
     def test_text_generation(self) -> None:
         """Tested separately in `test_inference_text_generation.py`."""
@@ -331,13 +376,13 @@ class InferenceClientVCRTest(InferenceClientTest):
 
     def test_translation(self) -> None:
         output = self.client.translation("Hello world")
-        self.assertEqual(output, "Hallo Welt")
+        assert output == TranslationOutput(translation_text="Hallo Welt", translation_output_translation_text=None)
 
     def test_translation_with_source_and_target_language(self) -> None:
         output_with_langs = self.client.translation(
             "Hello world", model="facebook/mbart-large-50-many-to-many-mmt", src_lang="en_XX", tgt_lang="fr_XX"
         )
-        self.assertIsInstance(output_with_langs, str)
+        assert isinstance(output_with_langs, TranslationOutput)
 
         with self.assertRaises(ValueError):
             self.client.translation("Hello world", model="facebook/mbart-large-50-many-to-many-mmt", src_lang="en_XX")
@@ -347,27 +392,24 @@ class InferenceClientVCRTest(InferenceClientTest):
 
     def test_token_classification(self) -> None:
         output = self.client.token_classification("My name is Sarah Jessica Parker but you can call me Jessica")
-        self.assertIsInstance(output, list)
-        self.assertGreater(len(output), 0)
-        for item in output:
-            self.assertIsInstance(item["entity_group"], str)
-            self.assertIsInstance(item["score"], float)
-            self.assertIsInstance(item["word"], str)
-            self.assertIsInstance(item["start"], int)
-            self.assertIsInstance(item["end"], int)
+        assert output == [
+            TokenClassificationOutputElement(
+                label=None, score=0.9991335868835449, end=31, entity_group="PER", start=11, word="Sarah Jessica Parker"
+            ),
+            TokenClassificationOutputElement(
+                label=None, score=0.9979913234710693, end=59, entity_group="PER", start=52, word="Jessica"
+            ),
+        ]
 
     def test_visual_question_answering(self) -> None:
         output = self.client.visual_question_answering(self.image_file, "Who's in the picture?")
-        self.assertEqual(
-            output,
-            [
-                {"score": 0.9386941194534302, "answer": "woman"},
-                {"score": 0.34311845898628235, "answer": "girl"},
-                {"score": 0.08407749235630035, "answer": "lady"},
-                {"score": 0.0507517009973526, "answer": "female"},
-                {"score": 0.01777094043791294, "answer": "man"},
-            ],
-        )
+        assert output == [
+            VisualQuestionAnsweringOutputElement(label=None, score=0.9386941194534302, answer="woman"),
+            VisualQuestionAnsweringOutputElement(label=None, score=0.34311845898628235, answer="girl"),
+            VisualQuestionAnsweringOutputElement(label=None, score=0.08407749235630035, answer="lady"),
+            VisualQuestionAnsweringOutputElement(label=None, score=0.0507517009973526, answer="female"),
+            VisualQuestionAnsweringOutputElement(label=None, score=0.01777094043791294, answer="man"),
+        ]
 
     def test_zero_shot_classification_single_label(self) -> None:
         output = self.client.zero_shot_classification(
@@ -395,24 +437,21 @@ class InferenceClientVCRTest(InferenceClientTest):
             labels=["space & cosmos", "scientific discovery", "microbiology", "robots", "archeology"],
             multi_label=True,
         )
-        self.assertEqual(
-            output,
-            [
-                {"label": "scientific discovery", "score": 0.9829297661781311},
-                {"label": "space & cosmos", "score": 0.755190908908844},
-                {"label": "microbiology", "score": 0.0005462635890580714},
-                {"label": "archeology", "score": 0.00047131875180639327},
-                {"label": "robots", "score": 0.00030448526376858354},
-            ],
-        )
+        assert output == [
+            ZeroShotClassificationOutputElement(label="scientific discovery", score=0.9829297661781311),
+            ZeroShotClassificationOutputElement(label="space & cosmos", score=0.755190908908844),
+            ZeroShotClassificationOutputElement(label="microbiology", score=0.0005462635890580714),
+            ZeroShotClassificationOutputElement(label="archeology", score=0.00047131875180639327),
+            ZeroShotClassificationOutputElement(label="robots", score=0.00030448526376858354),
+        ]
 
     def test_zero_shot_image_classification(self) -> None:
         output = self.client.zero_shot_image_classification(self.image_file, ["tree", "woman", "cat"])
         self.assertIsInstance(output, list)
         self.assertGreater(len(output), 0)
         for item in output:
-            self.assertIsInstance(item["label"], str)
-            self.assertIsInstance(item["score"], float)
+            self.assertIsInstance(item.label, str)
+            self.assertIsInstance(item.score, float)
 
     def test_unprocessable_entity_error(self) -> None:
         with self.assertRaisesRegex(HfHubHTTPError, "Make sure 'conversational' task is supported by the model."):
diff --git a/tests/test_inference_types.py b/tests/test_inference_types.py
new file mode 100644
index 00000000..0edc70b3
--- /dev/null
+++ b/tests/test_inference_types.py
@@ -0,0 +1,140 @@
+import json
+from dataclasses import dataclass
+from typing import List, Optional
+
+import pytest
+
+from huggingface_hub.inference._generated.types import BaseInferenceType
+
+from .testing_utils import expect_deprecation
+
+
+@dataclass
+class DummyType(BaseInferenceType):
+    foo: int
+    bar: str
+
+
+@dataclass
+class DummyNestedType(BaseInferenceType):
+    item: DummyType
+    items: List[DummyType]
+    maybe_items: Optional[List[DummyType]] = None
+
+
+DUMMY_AS_DICT = {"foo": 42, "bar": "baz"}
+DUMMY_AS_STR = json.dumps(DUMMY_AS_DICT)
+DUMMY_AS_BYTES = DUMMY_AS_STR.encode()
+DUMMY_AS_LIST = [DUMMY_AS_DICT]
+
+
+def test_parse_from_bytes():
+    instance = DummyType.parse_obj(DUMMY_AS_BYTES)
+    assert instance.foo == 42
+    assert instance.bar == "baz"
+
+
+def test_parse_from_str():
+    instance = DummyType.parse_obj(DUMMY_AS_STR)
+    assert instance.foo == 42
+    assert instance.bar == "baz"
+
+
+def test_parse_from_dict():
+    instance = DummyType.parse_obj(DUMMY_AS_DICT)
+    assert instance.foo == 42
+    assert instance.bar == "baz"
+
+
+def test_parse_from_list():
+    instances = DummyType.parse_obj(DUMMY_AS_LIST)
+    assert len(instances) == 1
+    assert instances[0].foo == 42
+    assert instances[0].bar == "baz"
+
+
+def test_parse_from_unexpected_type():
+    with pytest.raises(ValueError):
+        DummyType.parse_obj(42)
+
+
+def test_parse_as_instance_success():
+    instance = DummyType.parse_obj_as_instance(DUMMY_AS_DICT)
+    assert isinstance(instance, DummyType)
+
+
+def test_parse_as_instance_failure():
+    with pytest.raises(ValueError):
+        DummyType.parse_obj_as_instance(DUMMY_AS_LIST)
+
+
+def test_parse_as_list_success():
+    instances = DummyType.parse_obj_as_list(DUMMY_AS_LIST)
+    assert len(instances) == 1
+
+
+def test_parse_as_list_failure():
+    with pytest.raises(ValueError):
+        DummyType.parse_obj_as_list(DUMMY_AS_DICT)
+
+
+def test_parse_nested_class():
+    instance = DummyNestedType.parse_obj(
+        {
+            "item": DUMMY_AS_DICT,
+            "items": DUMMY_AS_LIST,
+            "maybe_items": None,
+        }
+    )
+    assert instance.item.foo == 42
+    assert instance.item.bar == "baz"
+    assert len(instance.items) == 1
+    assert instance.items[0].foo == 42
+    assert instance.items[0].bar == "baz"
+    assert instance.maybe_items is None
+
+
+def test_all_fields_are_optional():
+    # all fields are optional => silently accept None if server returns less data than expected
+    instance = DummyNestedType.parse_obj({"maybe_items": [{}, DUMMY_AS_BYTES]})
+    assert instance.item is None
+    assert instance.items is None
+    assert len(instance.maybe_items) == 2
+    assert instance.maybe_items[0].foo is None
+    assert instance.maybe_items[0].bar is None
+    assert instance.maybe_items[1].foo == 42
+    assert instance.maybe_items[1].bar == "baz"
+
+
+@expect_deprecation("DummyType")
+def test_not_expected_fields():
+    instance = DummyType.parse_obj({"foo": 42, "bar": "baz", "not_expected": "value"})
+    assert instance.foo == 42
+    assert instance.bar == "baz"
+    assert instance["not_expected"] == "value"
+
+
+@expect_deprecation("DummyType")
+def test_fields_kept_in_sync():
+    # hacky but works + will be removed once dataclasses will not be dicts anymore
+    instance = DummyType.parse_obj(DUMMY_AS_DICT)
+    assert instance.foo == 42
+    assert instance["foo"] == 42
+
+    instance.foo = 43
+    assert instance.foo == 43
+    assert instance["foo"] == 43
+
+    instance["foo"] = 44
+    assert instance.foo == 44
+    assert instance["foo"] == 44
+
+
+def test_normalize_keys():
+    # all fields are normalized in the dataclasses (by convention)
+    # if server response uses different keys, they will be normalized
+    instance = DummyNestedType.parse_obj({"ItEm": DUMMY_AS_DICT, "Maybe-Items": [DUMMY_AS_DICT]})
+    assert isinstance(instance.item, DummyType)
+    assert isinstance(instance.maybe_items, list)
+    assert len(instance.maybe_items) == 1
+    assert isinstance(instance.maybe_items[0], DummyType)