philipp-zettl/vrom-hf-docs huggingface dataset:
While there are such optional arguments as `tokenizer`, `feature_extractor`, `image_processor`, and `processor`,
they shouldn't be specified all at once. If these components are not provided, `pipeline` will try to load
required ones automatically. In case you want to provide these components explicitly, please refer to a
specific pipeline in order to get more details regarding what components are required.

Examples:

```python
>>> from transformers import pipeline, AutoModelForTokenClassification, AutoTokenizer

>>> # Sentiment analysis pipeline
>>> analyzer = pipeline("sentiment-analysis")

>>> # Named entity recognition pipeline, passing in a specific model and tokenizer
>>> model = AutoModelForTokenClassification.from_pretrained("dbmdz/bert-large-cased-finetuned-conll03-english")
>>> tokenizer = AutoTokenizer.from_pretrained("google-bert/bert-base-cased")
>>> recognizer = pipeline("ner", model=model, tokenizer=tokenizer)
```

**Parameters:**
model (`str` or [PreTrainedModel](/docs/transformers/v5.6.2/en/main_classes/model#transformers.PreTrainedModel), *optional*) : The model that will be used by the pipeline to make predictions. This can be a model identifier or an actual instance of a pretrained model inheriting from [PreTrainedModel](/docs/transformers/v5.6.2/en/main_classes/model#transformers.PreTrainedModel).  If not provided, the default for the `task` will be loaded.
config (`str` or [PreTrainedConfig](/docs/transformers/v5.6.2/en/main_classes/configuration#transformers.PreTrainedConfig), *optional*) : The configuration that will be used by the pipeline to instantiate the model. This can be a model identifier or an actual pretrained model configuration inheriting from [PreTrainedConfig](/docs/transformers/v5.6.2/en/main_classes/configuration#transformers.PreTrainedConfig).  If not provided, the default configuration file for the requested model will be used. That means that if `model` is given, its default configuration will be used. However, if `model` is not supplied, this `task`'s default model's config is used instead.
tokenizer (`str` or [PreTrainedTokenizer](/docs/transformers/v5.6.2/en/main_classes/tokenizer#transformers.PythonBackend), *optional*) : The tokenizer that will be used by the pipeline to encode data for the model. This can be a model identifier or an actual pretrained tokenizer inheriting from [PreTrainedTokenizer](/docs/transformers/v5.6.2/en/main_classes/tokenizer#transformers.PythonBackend).  If not provided, the default tokenizer for the given `model` will be loaded (if it is a string). If `model` is not specified or not a string, then the default tokenizer for `config` is loaded (if it is a string). However, if `config` is also not given or not a string, then the default tokenizer for the given `task` will be loaded.
feature_extractor (`str` or `PreTrainedFeatureExtractor`, *optional*) : The feature extractor that will be used by the pipeline to encode data for the model. This can be a model identifier or an actual pretrained feature extractor inheriting from `PreTrainedFeatureExtractor`.  Feature extractors are used for non-NLP models, such as Speech or Vision models as well as multi-modal models. Multi-modal models will also require a tokenizer to be passed.  If not provided, the default feature extractor for the given `model` will be loaded (if it is a string). If `model` is not specified or not a string, then the default feature extractor for `config` is loaded (if it is a string). However, if `config` is also not given or not a string, then the default feature extractor for the given `task` will be loaded.
image_processor (`str` or [BaseImageProcessor](/docs/transformers/v5.6.2/en/main_classes/image_processor#transformers.BaseImageProcessor), *optional*) : The image processor that will be used by the pipeline to preprocess images for the model. This can be a model identifier or an actual image processor inheriting from [BaseImageProcessor](/docs/transformers/v5.6.2/en/main_classes/image_processor#transformers.BaseImageProcessor).  Image processors are used for Vision models and multi-modal models that require image inputs. Multi-modal models will also require a tokenizer to be passed.  If not provided, the default image processor for the given `model` will be loaded (if it is a string). If `model` is not specified or not a string, then the default image processor for `config` is loaded (if it is a string).
processor (`str` or [ProcessorMixin](/docs/transformers/v5.6.2/en/main_classes/processors#transformers.ProcessorMixin), *optional*) : The processor that will be used by the pipeline to preprocess data for the model. This can be a model identifier or an actual processor inheriting from [ProcessorMixin](/docs/transformers/v5.6.2/en/main_classes/processors#transformers.ProcessorMixin).  Processors are used for multi-modal models that require multi-modal inputs, for example, a model that requires both text and image inputs.  If not provided, the default processor for the given `model` will be loaded (if it is a string). If `model` is not specified or not a string, then the default processor for `config` is loaded (if it is a string).
revision (`str`, *optional*, defaults to `"main"`) : When passing a task name or a string model identifier: The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any identifier allowed by git.

use_fast (`bool`, *optional*, defaults to `True`) : Whether or not to use a Fast tokenizer if possible (a [PreTrainedTokenizerFast](/docs/transformers/v5.6.2/en/main_classes/tokenizer#transformers.TokenizersBackend)).

token (`str` or *bool*, *optional*) : The token to use as HTTP bearer authorization for remote files. If `True`, will use the token generated when running `hf auth login`.

device (`int` or `str` or `torch.device`) : Defines the device (*e.g.*, `"cpu"`, `"cuda:1"`, `"mps"`, or a GPU ordinal rank like `1`) on which this pipeline will be allocated.
device_map (`str` or `dict[str, Union[int, str, torch.device]`, *optional*) : Sent directly as `model_kwargs` (just a simpler shortcut). When `accelerate` library is present, set `device_map="auto"` to compute the most optimized `device_map` automatically (see [here](https://huggingface.co/docs/accelerate/main/en/package_reference/big_modeling#accelerate.cpu_offload) for more information).    Do not use `device_map` AND `device` at the same time as they will conflict   

dtype (`str` or `torch.dtype`, *optional*) : Sent directly as `model_kwargs` (just a simpler shortcut) to use the available precision for this model (`torch.float16`, `torch.bfloat16`, ... or `"auto"`).
trust_remote_code (`bool`, *optional*, defaults to `False`) : Whether or not to allow for custom code defined on the Hub in their own modeling, configuration, tokenization or even pipeline files. This option should only be set to `True` for repositories you trust and in which you have read the code, as it will execute code present on the Hub on your local machine.

model_kwargs (`dict[str, Any]`, *optional*) : Additional dictionary of keyword arguments passed along to the model's `from_pretrained(..., **model_kwargs)` function.

kwargs (`dict[str, Any]`, *optional*) : Additional keyword arguments passed along to the specific pipeline init (see the documentation for the corresponding pipeline class for possible values).

**Returns:**

`[Pipeline](/docs/transformers/v5.6.2/en/main_classes/pipelines#transformers.Pipeline)`

A suitable pipeline for the task.
## Pipeline batching

All pipelines can use batching. This will work
whenever the pipeline uses its streaming ability (so when passing lists or `Dataset` or `generator`).

```python
from transformers import pipeline
from transformers.pipelines.pt_utils import KeyDataset
import datasets

dataset = datasets.load_dataset("imdb", name="plain_text", split="unsupervised")
pipe = pipeline("text-classification", device=0)
for out in pipe(KeyDataset(dataset, "text"), batch_size=8, truncation="only_first"):
    print(out)
    # [{'label': 'POSITIVE', 'score': 0.9998743534088135}]
    # Exactly the same output as before, but the content are passed
    # as batches to the model
```

However, this is not automatically a win for performance. It can be either a 10x speedup or 5x slowdown depending
on hardware, data and the actual model being used.

Example where it's mostly a speedup:
```python
from transformers import pipeline
from torch.utils.data import Dataset
from tqdm.auto import tqdm

pipe = pipeline("text-classification", device=0)

class MyDataset(Dataset):
    def __len__(self):
        return 5000

    def __getitem__(self, i):
        return "This is a test"

dataset = MyDataset()

for batch_size in [1, 8, 64, 256]:
    print("-" * 30)
    print(f"Streaming batch_size={batch_size}")
    for out in tqdm(pipe(dataset, batch_size=batch_size), total=len(dataset)):
        pass
```

```text
# On GTX 970
------------------------------
Streaming no batching
100%|██████████████████████████████████████████████████████████████████████| 5000/5000 [00:26
    for out in tqdm(pipe(dataset, batch_size=256), total=len(dataset)):
....
    q = q / math.sqrt(dim_per_head)  # (bs, n_heads, q_length, dim_per_head)
RuntimeError: CUDA out of memory. Tried to allocate 376.00 MiB (GPU 0; 3.95 GiB total capacity; 1.72 GiB already allocated; 354.88 MiB free; 2.46 GiB reserved in total by PyTorch)
```

There are no good (general) solutions for this problem, and your mileage may vary depending on your use cases. Rule of
thumb:

For users, a rule of thumb is:

- **Measure performance on your load, with your hardware. Measure, measure, and keep measuring. Real numbers are the
  only way to go.**
- If you are latency constrained (live product doing inference), don't batch.
- If you are using CPU, don't batch.
- If you are using throughput (you want to run your model on a bunch of static data), on GPU, then:
- If you have no clue about the size of the sequence_length ("natural" data), by default don't batch, measure and
    try tentatively to add it, add OOM checks to recover when it will fail (and it will at some point if you don't
    control the sequence_length.)
  - If your sequence_length is super regular, then batching is more likely to be VERY interesting, measure and push
    it until you get OOMs.
  - The larger the GPU the more likely batching is going to be more interesting
- As soon as you enable batching, make sure you can handle OOMs nicely.
## Pipeline chunk batching

`zero-shot-classification` and `question-answering` are slightly specific in the sense, that a single input might yield
multiple forward pass of a model. Under normal circumstances, this would yield issues with `batch_size` argument.

In order to circumvent this issue, both of these pipelines are a bit specific, they are `ChunkPipeline` instead of
regular `Pipeline`. In short:

```python
preprocessed = pipe.preprocess(inputs)
model_outputs = pipe.forward(preprocessed)
outputs = pipe.postprocess(model_outputs)
```

Now becomes:

```python
all_model_outputs = []
for preprocessed in pipe.preprocess(inputs):
    model_outputs = pipe.forward(preprocessed)
    all_model_outputs.append(model_outputs)
outputs = pipe.postprocess(all_model_outputs)
```

This should be very transparent to your code because the pipelines are used in
the same way.
## Pipeline FP16 inference

Models can be run in FP16 which can be significantly faster on GPU while saving memory. Most models will not suffer noticeable performance loss from this. The larger the model, the less likely that it will.

To enable FP16 inference, you can simply pass `dtype=torch.float16` or `dtype='float16'` to the pipeline constructor. Note that this only works for models with a PyTorch backend. Your inputs will be converted to FP16 internally.
## Pipeline custom code

If you want to override a specific pipeline.

Don't hesitate to create an issue for your task at hand, the goal of the pipeline is to be easy to use and support most
cases, so `transformers` could maybe support your use case.

If you want to try simply you can:

- Subclass your pipeline of choice

```python
class MyPipeline(TextClassificationPipeline):
    def postprocess():
        # Your code goes here
        scores = scores * 100
        # And here

my_pipeline = MyPipeline(model=model, tokenizer=tokenizer, ...)
#### transformers.AudioClassificationPipeline[[transformers.AudioClassificationPipeline]]

[Source](https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/pipelines/audio_classification.py#L67)

Audio classification pipeline using any `AutoModelForAudioClassification`. This pipeline predicts the class of a
raw waveform or an audio file. In case of an audio file, ffmpeg should be installed to support multiple audio
formats.

Example:

```python
>>> from transformers import pipeline

>>> classifier = pipeline(model="superb/wav2vec2-base-superb-ks")
>>> classifier("https://huggingface.co/datasets/Narsil/asr_dummy/resolve/main/1.flac")
[{'score': 0.997, 'label': '_unknown_'}, {'score': 0.002, 'label': 'left'}, {'score': 0.0, 'label': 'yes'}, {'score': 0.0, 'label': 'down'}, {'score': 0.0, 'label': 'stop'}]
```

Learn more about the basics of using a pipeline in the [pipeline tutorial](../pipeline_tutorial)
__call__transformers.AudioClassificationPipeline.__call__https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/pipelines/audio_classification.py#L109[{"name": "inputs", "val": ": numpy.ndarray | bytes | str | dict"}, {"name": "**kwargs", "val": ": typing.Any"}]- **inputs** (`np.ndarray` or `bytes` or `str` or `dict`) --
  The inputs is either :
  - `str` that is the filename of the audio file, the file will be read at the correct sampling rate
    to get the waveform using *ffmpeg*. This requires *ffmpeg* to be installed on the system.
  - `bytes` it is supposed to be the content of an audio file and is interpreted by *ffmpeg* in the
    same way.
  - (`np.ndarray` of shape (n, ) of type `np.float32` or `np.float64`)
    Raw audio at the correct sampling rate (no further check will be done)
  - `dict` form can be used to pass raw audio sampled at arbitrary `sampling_rate` and let this
    pipeline do the resampling. The dict must be either be in the format `{"sampling_rate": int,
    "raw": np.array}`, or `{"sampling_rate": int, "array": np.array}`, where the key `"raw"` or
    `"array"` is used to denote the raw audio waveform.
- **top_k** (`int`, *optional*, defaults to None) --
  The number of top labels that will be returned by the pipeline. If the provided number is `None` or
  higher than the number of labels available in the model configuration, it will default to the number of
  labels.
- **function_to_apply** (`str`, *optional*, defaults to "softmax") --
  The function to apply to the model output. By default, the pipeline will apply the softmax function to
  the output of the model. Valid options: ["softmax", "sigmoid", "none"]. Note that passing Python's
  built-in `None` will default to "softmax", so you need to pass the string "none" to disable any
  post-processing.0A list of `dict` with the following keys- **label** (`str`) -- The label predicted.
- **score** (`float`) -- The corresponding probability.
Classify the sequence(s) given as inputs. See the [AutomaticSpeechRecognitionPipeline](/docs/transformers/v5.6.2/en/main_classes/pipelines#transformers.AutomaticSpeechRecognitionPipeline) documentation for more
information.

**Parameters:**

model ([PreTrainedModel](/docs/transformers/v5.6.2/en/main_classes/model#transformers.PreTrainedModel)) : The model that will be used by the pipeline to make predictions. This needs to be a model inheriting from [PreTrainedModel](/docs/transformers/v5.6.2/en/main_classes/model#transformers.PreTrainedModel).

feature_extractor ([SequenceFeatureExtractor](/docs/transformers/v5.6.2/en/main_classes/feature_extractor#transformers.SequenceFeatureExtractor)) : The feature extractor that will be used by the pipeline to encode data for the model. This object inherits from [SequenceFeatureExtractor](/docs/transformers/v5.6.2/en/main_classes/feature_extractor#transformers.SequenceFeatureExtractor).

task (`str`, defaults to `""`) : A task-identifier for the pipeline.
num_workers (`int`, *optional*, defaults to 8) : When the pipeline will use *DataLoader* (when passing a dataset, on GPU for a Pytorch model), the number of workers to be used.

batch_size (`int`, *optional*, defaults to 1) : When the pipeline will use *DataLoader* (when passing a dataset, on GPU for a Pytorch model), the size of the batch to use, for inference this is not always beneficial, please read [Batching with pipelines](https://huggingface.co/transformers/main_classes/pipelines.html#pipeline-batching) .

args_parser ([ArgumentHandler](/docs/transformers/v5.6.2/en/internal/pipelines_utils#transformers.pipelines.ArgumentHandler), *optional*) : Reference to the object in charge of parsing supplied pipeline parameters.

device (`int`, *optional*, defaults to -1) : Device ordinal for CPU/GPU supports. Setting this to -1 will leverage CPU, a positive will run the model on the associated CUDA device id. You can pass native `torch.device` or a `str` too
dtype (`str` or `torch.dtype`, *optional*) : Sent directly as `model_kwargs` (just a simpler shortcut) to use the available precision for this model (`torch.float16`, `torch.bfloat16`, ... or `"auto"`)

binary_output (`bool`, *optional*, defaults to `False`) : Flag indicating if the output the pipeline should happen in a serialized format (i.e., pickle) or as the raw output data e.g. text.

**Returns:**

`A list of `dict` with the following keys`

- **label** (`str`) -- The label predicted.
- **score** (`float`) -- The corresponding probability.
#### transformers.AutomaticSpeechRecognitionPipeline[[transformers.AutomaticSpeechRecognitionPipeline]]

[Source](https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/pipelines/automatic_speech_recognition.py#L112)

Pipeline that aims at extracting spoken text contained within some audio.

The input can be either a raw waveform or a audio file. In case of the audio file, ffmpeg should be installed for
to support multiple audio formats

Unless the model you're using explicitly sets these generation parameters in its configuration files
(`generation_config.json`), the following default values will be used:
- max_new_tokens: 256
- num_beams: 5

Example:
```python
>>> from transformers import pipeline

>>> transcriber = pipeline(model="openai/whisper-base")
>>> transcriber("https://huggingface.co/datasets/Narsil/asr_dummy/resolve/main/1.flac")
{'text': ' He hoped there would be stew for dinner, turnips and carrots and bruised potatoes and fat mutton pieces to be ladled out in thick, peppered flour-fatten sauce.'}
```

Learn more about the basics of using a pipeline in the [pipeline tutorial](../pipeline_tutorial)
__call__transformers.AutomaticSpeechRecognitionPipeline.__call__https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/pipelines/automatic_speech_recognition.py#L187[{"name": "inputs", "val": ": numpy.ndarray | bytes | str | dict"}, {"name": "**kwargs", "val": ": typing.Any"}]- **inputs** (`np.ndarray` or `bytes` or `str` or `dict`) --
  The inputs is either :
  - `str` that is either the filename of a local audio file, or a public URL address to download the
    audio file. The file will be read at the correct sampling rate to get the waveform using
    *ffmpeg*. This requires *ffmpeg* to be installed on the system.
  - `bytes` it is supposed to be the content of an audio file and is interpreted by *ffmpeg* in the
    same way.
  - (`np.ndarray` of shape (n, ) of type `np.float32` or `np.float64`)
    Raw audio at the correct sampling rate (no further check will be done)
  - `dict` form can be used to pass raw audio sampled at arbitrary `sampling_rate` and let this
    pipeline do the resampling. The dict must be in the format `{"sampling_rate": int, "raw":
    np.array}` with optionally a `"stride": (left: int, right: int)` than can ask the pipeline to
    treat the first `left` samples and last `right` samples to be ignored in decoding (but used at
    inference to provide more context to the model). Only use `stride` with CTC models.
- **return_timestamps** (*optional*, `str` or `bool`) --
  Only available for pure CTC models (Wav2Vec2, HuBERT, etc) and the Whisper model. Not available for
  other sequence-to-sequence models.
For CTC models, timestamps can take one of two formats:
  - `"char"`: the pipeline will return timestamps along the text for every character in the text. For
    instance, if you get `[{"text": "h", "timestamp": (0.5, 0.6)}, {"text": "i", "timestamp": (0.7,
    0.9)}]`, then it means the model predicts that the letter "h" was spoken after `0.5` and before
    `0.6` seconds.
  - `"word"`: the pipeline will return timestamps along the text for every word in the text. For
    instance, if you get `[{"text": "hi ", "timestamp": (0.5, 0.9)}, {"text": "there", "timestamp":
    (1.0, 1.5)}]`, then it means the model predicts that the word "hi" was spoken after `0.5` and
    before `0.9` seconds.
For the Whisper model, timestamps can take one of two formats:
  - `"word"`: same as above for word-level CTC timestamps. Word-level timestamps are predicted
    through the *dynamic-time warping (DTW)* algorithm, an approximation to word-level timestamps
    by inspecting the cross-attention weights.
  - `True`: the pipeline will return timestamps along the text for *segments* of words in the text.
    For instance, if you get `[{"text": " Hi there!", "timestamp": (0.5, 1.5)}]`, then it means the
    model predicts that the segment "Hi there!" was spoken after `0.5` and before `1.5` seconds.
    Note that a segment of text refers to a sequence of one or more words, rather than individual
    words as with word-level timestamps.
- **generate_kwargs** (`dict`, *optional*) --
  The dictionary of ad-hoc parametrization of `generate_config` to be used for the generation call. For a
  complete overview of generate, check the [following
  guide](https://huggingface.co/docs/transformers/en/main_classes/text_generation).0`Dict`A dictionary with the following keys:
- **text** (`str`): The recognized text.
- **chunks** (*optional(, `list[Dict]`)
  When using `return_timestamps`, the `chunks` will become a list containing all the various text
  chunks identified by the model, *e.g.* `[{"text": "hi ", "timestamp": (0.5, 0.9)}, {"text":
  "there", "timestamp": (1.0, 1.5)}]`. The original full text can roughly be recovered by doing
  `"".join(chunk["text"] for chunk in output["chunks"])`.
Transcribe the audio sequence(s) given as inputs to text. See the [AutomaticSpeechRecognitionPipeline](/docs/transformers/v5.6.2/en/main_classes/pipelines#transformers.AutomaticSpeechRecognitionPipeline)
documentation for more information.

**Parameters:**

model ([PreTrainedModel](/docs/transformers/v5.6.2/en/main_classes/model#transformers.PreTrainedModel)) : The model that will be used by the pipeline to make predictions. This needs to be a model inheriting from [PreTrainedModel](/docs/transformers/v5.6.2/en/main_classes/model#transformers.PreTrainedModel).

feature_extractor ([SequenceFeatureExtractor](/docs/transformers/v5.6.2/en/main_classes/feature_extractor#transformers.SequenceFeatureExtractor), *optional*) : The feature extractor that will be used by the pipeline to encode waveform for the model.