philipp-zettl/vrom-hf-docs huggingface dataset:
tokenizer ([PreTrainedTokenizer](/docs/transformers/v5.6.2/en/main_classes/tokenizer#transformers.PythonBackend), *optional*) : The tokenizer that will be used by the pipeline to encode data for the model. This object inherits from [PreTrainedTokenizer](/docs/transformers/v5.6.2/en/main_classes/tokenizer#transformers.PythonBackend).

decoder (`pyctcdecode.BeamSearchDecoderCTC`, *optional*) : [PyCTCDecode's BeamSearchDecoderCTC](https://github.com/kensho-technologies/pyctcdecode/blob/2fd33dc37c4111417e08d89ccd23d28e9b308d19/pyctcdecode/decoder.py#L180) can be passed for language model boosted decoding. See [Wav2Vec2ProcessorWithLM](/docs/transformers/v5.6.2/en/model_doc/wav2vec2#transformers.Wav2Vec2ProcessorWithLM) for more information.

device (Union[`int`, `torch.device`], *optional*) : Device ordinal for CPU/GPU supports. Setting this to `None` will leverage CPU, a positive will run the model on the associated CUDA device id.

**Returns:**

``Dict``
A dictionary with the following keys:
- **text** (`str`): The recognized text.
- **chunks** (*optional(, `list[Dict]`)
  When using `return_timestamps`, the `chunks` will become a list containing all the various text
  chunks identified by the model, *e.g.* `[{"text": "hi ", "timestamp": (0.5, 0.9)}, {"text":
  "there", "timestamp": (1.0, 1.5)}]`. The original full text can roughly be recovered by doing
  `"".join(chunk["text"] for chunk in output["chunks"])`.
#### transformers.TextToAudioPipeline[[transformers.TextToAudioPipeline]]

[Source](https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/pipelines/text_to_audio.py#L45)

Text-to-audio generation pipeline using any `AutoModelForTextToWaveform` or `AutoModelForTextToSpectrogram`. This
pipeline generates an audio file from an input text and optional other conditional inputs.

Unless the model you're using explicitly sets these generation parameters in its configuration files
(`generation_config.json`), the following default values will be used:
- max_new_tokens: 256

Example:

```python
>>> from transformers import pipeline

>>> pipe = pipeline(model="suno/bark-small")
>>> output = pipe("Hey it's HuggingFace on the phone!")

>>> audio = output["audio"]
>>> sampling_rate = output["sampling_rate"]
```

Learn more about the basics of using a pipeline in the [pipeline tutorial](../pipeline_tutorial)
You can specify parameters passed to the model by using `TextToAudioPipeline.__call__.forward_params` or
`TextToAudioPipeline.__call__.generate_kwargs`.

Example:

```python
>>> from transformers import pipeline

>>> music_generator = pipeline(task="text-to-audio", model="facebook/musicgen-small")

>>> # diversify the music generation by adding randomness with a high temperature and set a maximum music length
>>> generate_kwargs = {
...     "do_sample": True,
...     "temperature": 0.7,
...     "max_new_tokens": 35,
... }

>>> outputs = music_generator("Techno music with high melodic riffs", generate_kwargs=generate_kwargs)
```

This pipeline can currently be loaded from [pipeline()](/docs/transformers/v5.6.2/en/main_classes/pipelines#transformers.pipeline) using the following task identifiers: `"text-to-speech"` or
`"text-to-audio"`.

See the list of available models on [huggingface.co/models](https://huggingface.co/models?filter=text-to-speech).
__call__transformers.TextToAudioPipeline.__call__https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/pipelines/text_to_audio.py#L243[{"name": "text_inputs", "val": ""}, {"name": "**forward_params", "val": ""}]- **text_inputs** (`str`, `list[str]`, `ChatType`, or `list[ChatType]`) --
  One or several texts to generate. If strings or a list of string are passed, this pipeline will
  generate the corresponding text. Alternatively, a "chat", in the form of a list of dicts with "role"
  and "content" keys, can be passed, or a list of such chats. When chats are passed, the model's chat
  template will be used to format them before passing them to the model.
- **forward_params** (`dict`, *optional*) --
  Parameters passed to the model generation/forward method. `forward_params` are always passed to the
  underlying model.
- **generate_kwargs** (`dict`, *optional*) --
  The dictionary of ad-hoc parametrization of `generate_config` to be used for the generation call. For a
  complete overview of generate, check the [following
  guide](https://huggingface.co/docs/transformers/en/main_classes/text_generation). `generate_kwargs` are
  only passed to the underlying model if the latter is a generative model.0`AudioOutput` or a list of `AudioOutput`, which is a `TypedDict` with two keys- **audio** (`np.ndarray` of shape `(nb_channels, audio_length)`) -- The generated audio waveform.
- **sampling_rate** (`int`) -- The sampling rate of the generated audio waveform.
Generates speech/audio from the inputs. See the [TextToAudioPipeline](/docs/transformers/v5.6.2/en/main_classes/pipelines#transformers.TextToAudioPipeline) documentation for more information.

**Parameters:**

text_inputs (`str`, `list[str]`, `ChatType`, or `list[ChatType]`) : One or several texts to generate. If strings or a list of string are passed, this pipeline will generate the corresponding text. Alternatively, a "chat", in the form of a list of dicts with "role" and "content" keys, can be passed, or a list of such chats. When chats are passed, the model's chat template will be used to format them before passing them to the model.

forward_params (`dict`, *optional*) : Parameters passed to the model generation/forward method. `forward_params` are always passed to the underlying model.
generate_kwargs (`dict`, *optional*) : The dictionary of ad-hoc parametrization of `generate_config` to be used for the generation call. For a complete overview of generate, check the [following guide](https://huggingface.co/docs/transformers/en/main_classes/text_generation). `generate_kwargs` are only passed to the underlying model if the latter is a generative model.

**Returns:**

``AudioOutput` or a list of `AudioOutput`, which is a `TypedDict` with two keys`

- **audio** (`np.ndarray` of shape `(nb_channels, audio_length)`) -- The generated audio waveform.
- **sampling_rate** (`int`) -- The sampling rate of the generated audio waveform.
#### transformers.ZeroShotAudioClassificationPipeline[[transformers.ZeroShotAudioClassificationPipeline]]

[Source](https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/pipelines/zero_shot_audio_classification.py#L32)

Zero shot audio classification pipeline using `ClapModel`. This pipeline predicts the class of an audio when you
provide an audio and a set of `candidate_labels`.

The default `hypothesis_template` is : `"This is a sound of {}."`. Make sure you update it for your usage.

Example:
```python
>>> from transformers import pipeline
>>> from datasets import load_dataset

>>> dataset = load_dataset("ashraq/esc50")
>>> audio = next(iter(dataset["train"]["audio"]))["array"]
>>> classifier = pipeline(task="zero-shot-audio-classification", model="laion/clap-htsat-unfused")
>>> classifier(audio, candidate_labels=["Sound of a dog", "Sound of vacuum cleaner"])
[{'score': 0.9996, 'label': 'Sound of a dog'}, {'score': 0.0004, 'label': 'Sound of vacuum cleaner'}]
```
Learn more about the basics of using a pipeline in the [pipeline tutorial](../pipeline_tutorial) This audio
classification pipeline can currently be loaded from [pipeline()](/docs/transformers/v5.6.2/en/main_classes/pipelines#transformers.pipeline) using the following task identifier:
`"zero-shot-audio-classification"`. See the list of available models on
[huggingface.co/models](https://huggingface.co/models?filter=zero-shot-audio-classification).
__call__transformers.ZeroShotAudioClassificationPipeline.__call__https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/pipelines/zero_shot_audio_classification.py#L70[{"name": "audios", "val": ": numpy.ndarray | bytes | str | dict"}, {"name": "**kwargs", "val": ": typing.Any"}]- **audios** (`str`, `list[str]`, `np.array` or `list[np.array]`) --
  The pipeline handles three types of inputs:
  - A string containing a http link pointing to an audio
  - A string containing a local path to an audio
  - An audio loaded in numpy
- **candidate_labels** (`list[str]`) --
  The candidate labels for this audio. They will be formatted using *hypothesis_template*.
- **hypothesis_template** (`str`, *optional*, defaults to `"This is a sound of {}"`) --
  The format used in conjunction with *candidate_labels* to attempt the audio classification by
  replacing the placeholder with the candidate_labels. Pass "{}" if *candidate_labels* are
  already formatted.0A list of dictionaries containing one entry per proposed label. Each dictionary contains the
following keys:
- **label** (`str`) -- One of the suggested *candidate_labels*.
- **score** (`float`) -- The score attributed by the model to that label. It is a value between
  0 and 1, computed as the `softmax` of `logits_per_audio`.
Assign labels to the audio(s) passed as inputs.

**Parameters:**

model ([PreTrainedModel](/docs/transformers/v5.6.2/en/main_classes/model#transformers.PreTrainedModel)) : The model that will be used by the pipeline to make predictions. This needs to be a model inheriting from [PreTrainedModel](/docs/transformers/v5.6.2/en/main_classes/model#transformers.PreTrainedModel).

tokenizer ([PreTrainedTokenizer](/docs/transformers/v5.6.2/en/main_classes/tokenizer#transformers.PythonBackend)) : The tokenizer that will be used by the pipeline to encode data for the model. This object inherits from [PreTrainedTokenizer](/docs/transformers/v5.6.2/en/main_classes/tokenizer#transformers.PythonBackend).
feature_extractor ([SequenceFeatureExtractor](/docs/transformers/v5.6.2/en/main_classes/feature_extractor#transformers.SequenceFeatureExtractor)) : The feature extractor that will be used by the pipeline to encode data for the model. This object inherits from [SequenceFeatureExtractor](/docs/transformers/v5.6.2/en/main_classes/feature_extractor#transformers.SequenceFeatureExtractor).

task (`str`, defaults to `""`) : A task-identifier for the pipeline.

num_workers (`int`, *optional*, defaults to 8) : When the pipeline will use *DataLoader* (when passing a dataset, on GPU for a Pytorch model), the number of workers to be used.

batch_size (`int`, *optional*, defaults to 1) : When the pipeline will use *DataLoader* (when passing a dataset, on GPU for a Pytorch model), the size of the batch to use, for inference this is not always beneficial, please read [Batching with pipelines](https://huggingface.co/transformers/main_classes/pipelines.html#pipeline-batching) .
args_parser ([ArgumentHandler](/docs/transformers/v5.6.2/en/internal/pipelines_utils#transformers.pipelines.ArgumentHandler), *optional*) : Reference to the object in charge of parsing supplied pipeline parameters.

device (`int`, *optional*, defaults to -1) : Device ordinal for CPU/GPU supports. Setting this to -1 will leverage CPU, a positive will run the model on the associated CUDA device id. You can pass native `torch.device` or a `str` too

dtype (`str` or `torch.dtype`, *optional*) : Sent directly as `model_kwargs` (just a simpler shortcut) to use the available precision for this model (`torch.float16`, `torch.bfloat16`, ... or `"auto"`)

binary_output (`bool`, *optional*, defaults to `False`) : Flag indicating if the output the pipeline should happen in a serialized format (i.e., pickle) or as the raw output data e.g. text.

**Returns:**
#### transformers.DepthEstimationPipeline[[transformers.DepthEstimationPipeline]]

[Source](https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/pipelines/depth_estimation.py#L25)

Depth estimation pipeline using any `AutoModelForDepthEstimation`. This pipeline predicts the depth of an image.

Example:

```python
>>> from transformers import pipeline

>>> depth_estimator = pipeline(task="depth-estimation", model="LiheYoung/depth-anything-base-hf")
>>> output = depth_estimator("http://images.cocodataset.org/val2017/000000039769.jpg")
>>> # This is a tensor with the values being the depth expressed in meters for each pixel
>>> output["predicted_depth"].shape
torch.Size([1, 384, 384])
```

Learn more about the basics of using a pipeline in the [pipeline tutorial](../pipeline_tutorial)

This depth estimation pipeline can currently be loaded from [pipeline()](/docs/transformers/v5.6.2/en/main_classes/pipelines#transformers.pipeline) using the following task identifier:
`"depth-estimation"`.
See the list of available models on [huggingface.co/models](https://huggingface.co/models?filter=depth-estimation).

__call__transformers.DepthEstimationPipeline.__call__https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/pipelines/depth_estimation.py#L66[{"name": "inputs", "val": ": typing.Union[str, list[str], ForwardRef('Image.Image'), list['Image.Image']]"}, {"name": "**kwargs", "val": ": typing.Any"}]- **inputs** (`str`, `list[str]`, `PIL.Image` or `list[PIL.Image]`) --
  The pipeline handles three types of images:

  - A string containing a http link pointing to an image
  - A string containing a local path to an image
  - An image loaded in PIL directly
The pipeline accepts either a single image or a batch of images, which must then be passed as a string.
  Images in a batch must all be in the same format: all as http links, all as local paths, or all as PIL
  images.
- **parameters** (`Dict`, *optional*) --
  A dictionary of argument names to parameter values, to control pipeline behaviour.
  The only parameter available right now is `timeout`, which is the length of time, in seconds,
  that the pipeline should wait before giving up on trying to download an image.
- **timeout** (`float`, *optional*, defaults to None) --
  The maximum time in seconds to wait for fetching images from the web. If None, no timeout is set and
  the call may block forever.0A dictionary or a list of dictionaries containing result. If the input is a single image, will return a
dictionary, if the input is a list of several images, will return a list of dictionaries corresponding to
the images.

The dictionaries contain the following keys:
- **predicted_depth** (`torch.Tensor`) -- The predicted depth by the model as a `torch.Tensor`.
- **depth** (`PIL.Image`) -- The predicted depth by the model as a `PIL.Image`.

Predict the depth(s) of the image(s) passed as inputs.

**Parameters:**

model ([PreTrainedModel](/docs/transformers/v5.6.2/en/main_classes/model#transformers.PreTrainedModel)) : The model that will be used by the pipeline to make predictions. This needs to be a model inheriting from [PreTrainedModel](/docs/transformers/v5.6.2/en/main_classes/model#transformers.PreTrainedModel).

image_processor ([BaseImageProcessor](/docs/transformers/v5.6.2/en/main_classes/image_processor#transformers.BaseImageProcessor)) : The image processor that will be used by the pipeline to encode data for the model. This object inherits from [BaseImageProcessor](/docs/transformers/v5.6.2/en/main_classes/image_processor#transformers.BaseImageProcessor).

task (`str`, defaults to `""`) : A task-identifier for the pipeline.
dtype (`str` or `torch.dtype`, *optional*) : Sent directly as `model_kwargs` (just a simpler shortcut) to use the available precision for this model (`torch.float16`, `torch.bfloat16`, ... or `"auto"`)

binary_output (`bool`, *optional*, defaults to `False`) : Flag indicating if the output the pipeline should happen in a serialized format (i.e., pickle) or as the raw output data e.g. text.

**Returns:**

A dictionary or a list of dictionaries containing result. If the input is a single image, will return a
dictionary, if the input is a list of several images, will return a list of dictionaries corresponding to
the images.

The dictionaries contain the following keys:

- **predicted_depth** (`torch.Tensor`) -- The predicted depth by the model as a `torch.Tensor`.
- **depth** (`PIL.Image`) -- The predicted depth by the model as a `PIL.Image`.
#### transformers.ImageClassificationPipeline[[transformers.ImageClassificationPipeline]]

[Source](https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/pipelines/image_classification.py#L73)

Image classification pipeline using any `AutoModelForImageClassification`. This pipeline predicts the class of an
image.

Example:

```python
>>> from transformers import pipeline

>>> classifier = pipeline(model="microsoft/beit-base-patch16-224-pt22k-ft22k")
>>> classifier("https://huggingface.co/datasets/Narsil/image_dummy/raw/main/parrots.png")
[{'score': 0.442, 'label': 'macaw'}, {'score': 0.088, 'label': 'popinjay'}, {'score': 0.075, 'label': 'parrot'}, {'score': 0.073, 'label': 'parodist, lampooner'}, {'score': 0.046, 'label': 'poll, poll_parrot'}]
```

Learn more about the basics of using a pipeline in the [pipeline tutorial](../pipeline_tutorial)
This image classification pipeline can currently be loaded from [pipeline()](/docs/transformers/v5.6.2/en/main_classes/pipelines#transformers.pipeline) using the following task identifier:
`"image-classification"`.

See the list of available models on
[huggingface.co/models](https://huggingface.co/models?filter=image-classification).

__call__transformers.ImageClassificationPipeline.__call__https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/pipelines/image_classification.py#L127[{"name": "inputs", "val": ": typing.Union[str, list[str], ForwardRef('Image.Image'), list['Image.Image']]"}, {"name": "**kwargs", "val": ": typing.Any"}]- **inputs** (`str`, `list[str]`, `PIL.Image` or `list[PIL.Image]`) --
  The pipeline handles three types of images:

  - A string containing a http link pointing to an image
  - A string containing a local path to an image
  - An image loaded in PIL directly
The pipeline accepts either a single image or a batch of images, which must then be passed as a string.
  Images in a batch must all be in the same format: all as http links, all as local paths, or all as PIL
  images.
- **function_to_apply** (`str`, *optional*, defaults to `"default"`) --
  The function to apply to the model outputs in order to retrieve the scores. Accepts four different
  values:

  If this argument is not specified, then it will apply the following functions according to the number
  of labels:

  - If the model has a single label, will apply the sigmoid function on the output.
  - If the model has several labels, will apply the softmax function on the output.

  Possible values are:
- `"sigmoid"`: Applies the sigmoid function on the output.
  - `"softmax"`: Applies the softmax function on the output.
  - `"none"`: Does not apply any function on the output.
- **top_k** (`int`, *optional*, defaults to 5) --
  The number of top labels that will be returned by the pipeline. If the provided number is higher than
  the number of labels available in the model configuration, it will default to the number of labels.
- **timeout** (`float`, *optional*, defaults to None) --
  The maximum time in seconds to wait for fetching images from the web. If None, no timeout is set and
  the call may block forever.0A dictionary or a list of dictionaries containing result. If the input is a single image, will return a
dictionary, if the input is a list of several images, will return a list of dictionaries corresponding to
the images.

The dictionaries contain the following keys:

- **label** (`str`) -- The label identified by the model.
- **score** (`int`) -- The score attributed by the model for that label.
Assign labels to the image(s) passed as inputs.

**Parameters:**

model ([PreTrainedModel](/docs/transformers/v5.6.2/en/main_classes/model#transformers.PreTrainedModel)) : The model that will be used by the pipeline to make predictions. This needs to be a model inheriting from [PreTrainedModel](/docs/transformers/v5.6.2/en/main_classes/model#transformers.PreTrainedModel).

image_processor ([BaseImageProcessor](/docs/transformers/v5.6.2/en/main_classes/image_processor#transformers.BaseImageProcessor)) : The image processor that will be used by the pipeline to encode data for the model. This object inherits from [BaseImageProcessor](/docs/transformers/v5.6.2/en/main_classes/image_processor#transformers.BaseImageProcessor).

task (`str`, defaults to `""`) : A task-identifier for the pipeline.

num_workers (`int`, *optional*, defaults to 8) : When the pipeline will use *DataLoader* (when passing a dataset, on GPU for a Pytorch model), the number of workers to be used.
batch_size (`int`, *optional*, defaults to 1) : When the pipeline will use *DataLoader* (when passing a dataset, on GPU for a Pytorch model), the size of the batch to use, for inference this is not always beneficial, please read [Batching with pipelines](https://huggingface.co/transformers/main_classes/pipelines.html#pipeline-batching) .

args_parser ([ArgumentHandler](/docs/transformers/v5.6.2/en/internal/pipelines_utils#transformers.pipelines.ArgumentHandler), *optional*) : Reference to the object in charge of parsing supplied pipeline parameters.

device (`int`, *optional*, defaults to -1) : Device ordinal for CPU/GPU supports. Setting this to -1 will leverage CPU, a positive will run the model on the associated CUDA device id. You can pass native `torch.device` or a `str` too

dtype (`str` or `torch.dtype`, *optional*) : Sent directly as `model_kwargs` (just a simpler shortcut) to use the available precision for this model (`torch.float16`, `torch.bfloat16`, ... or `"auto"`)
binary_output (`bool`, *optional*, defaults to `False`) : Flag indicating if the output the pipeline should happen in a serialized format (i.e., pickle) or as the raw output data e.g. text.

function_to_apply (`str`, *optional*, defaults to `"default"`) : The function to apply to the model outputs in order to retrieve the scores. Accepts four different values:  - `"default"`: if the model has a single label, will apply the sigmoid function on the output. If the model has several labels, will apply the softmax function on the output. - `"sigmoid"`: Applies the sigmoid function on the output. - `"softmax"`: Applies the softmax function on the output. - `"none"`: Does not apply any function on the output.

**Returns:**

A dictionary or a list of dictionaries containing result. If the input is a single image, will return a
dictionary, if the input is a list of several images, will return a list of dictionaries corresponding to
the images.

The dictionaries contain the following keys:
#### transformers.ImageSegmentationPipeline[[transformers.ImageSegmentationPipeline]]

[Source](https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/pipelines/image_segmentation.py#L27)

Image segmentation pipeline using any `AutoModelForXXXSegmentation`. This pipeline predicts masks of objects and
their classes.

Example:

```python
>>> from transformers import pipeline

>>> segmenter = pipeline(model="facebook/detr-resnet-50-panoptic")
>>> segments = segmenter("https://huggingface.co/datasets/Narsil/image_dummy/raw/main/parrots.png")
>>> len(segments)
2

>>> segments[0]["label"]
'bird'

>>> segments[1]["label"]
'bird'

>>> type(segments[0]["mask"])  # This is a black and white mask showing where is the bird on the original image.

>>> segments[0]["mask"].size
(768, 512)
```

This image segmentation pipeline can currently be loaded from [pipeline()](/docs/transformers/v5.6.2/en/main_classes/pipelines#transformers.pipeline) using the following task identifier:
`"image-segmentation"`.
See the list of available models on
[huggingface.co/models](https://huggingface.co/models?filter=image-segmentation).

__call__transformers.ImageSegmentationPipeline.__call__https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/pipelines/image_segmentation.py#L101[{"name": "inputs", "val": ": typing.Union[str, ForwardRef('Image.Image'), list[str], list['Image.Image']]"}, {"name": "**kwargs", "val": ": typing.Any"}]- **inputs** (`str`, `list[str]`, `PIL.Image` or `list[PIL.Image]`) --
  The pipeline handles three types of images:

  - A string containing an HTTP(S) link pointing to an image
  - A string containing a local path to an image
  - An image loaded in PIL directly