philipp-zettl/vrom-hf-docs huggingface dataset:
The pipeline accepts either a single image or a batch of images. Images in a batch must all be in the
  same format: all as HTTP(S) links, all as local paths, or all as PIL images.
- **subtask** (`str`, *optional*) --
  Segmentation task to be performed, choose [`semantic`, `instance` and `panoptic`] depending on model
  capabilities. If not set, the pipeline will attempt tp resolve in the following order:
  `panoptic`, `instance`, `semantic`.
- **threshold** (`float`, *optional*, defaults to 0.9) --
  Probability threshold to filter out predicted masks.
- **mask_threshold** (`float`, *optional*, defaults to 0.5) --
  Threshold to use when turning the predicted masks into binary values.
- **overlap_mask_area_threshold** (`float`, *optional*, defaults to 0.5) --
  Mask overlap threshold to eliminate small, disconnected segments.
- **timeout** (`float`, *optional*, defaults to None) --
  The maximum time in seconds to wait for fetching images from the web. If None, no timeout is set and
  the call may block forever.0If the input is a single image, will return a list of dictionaries, if the input is a list of several images,
will return a list of list of dictionaries corresponding to each image.
The dictionaries contain the mask, label and score (where applicable) of each detected object and contains
the following keys:

- **label** (`str`) -- The class label identified by the model.
- **mask** (`PIL.Image`) -- A binary mask of the detected object as a Pil Image of shape (width, height) of
  the original image. Returns a mask filled with zeros if no object is found.
- **score** (*optional* `float`) -- Optionally, when the model is capable of estimating a confidence of the
  "object" described by the label and the mask.

Perform segmentation (detect masks & classes) in the image(s) passed as inputs.

**Parameters:**

model ([PreTrainedModel](/docs/transformers/v5.6.2/en/main_classes/model#transformers.PreTrainedModel)) : The model that will be used by the pipeline to make predictions. This needs to be a model inheriting from [PreTrainedModel](/docs/transformers/v5.6.2/en/main_classes/model#transformers.PreTrainedModel).
image_processor ([BaseImageProcessor](/docs/transformers/v5.6.2/en/main_classes/image_processor#transformers.BaseImageProcessor)) : The image processor that will be used by the pipeline to encode data for the model. This object inherits from [BaseImageProcessor](/docs/transformers/v5.6.2/en/main_classes/image_processor#transformers.BaseImageProcessor).

task (`str`, defaults to `""`) : A task-identifier for the pipeline.

num_workers (`int`, *optional*, defaults to 8) : When the pipeline will use *DataLoader* (when passing a dataset, on GPU for a Pytorch model), the number of workers to be used.

batch_size (`int`, *optional*, defaults to 1) : When the pipeline will use *DataLoader* (when passing a dataset, on GPU for a Pytorch model), the size of the batch to use, for inference this is not always beneficial, please read [Batching with pipelines](https://huggingface.co/transformers/main_classes/pipelines.html#pipeline-batching) .
If the input is a single image, will return a list of dictionaries, if the input is a list of several images,
will return a list of list of dictionaries corresponding to each image.

The dictionaries contain the mask, label and score (where applicable) of each detected object and contains
the following keys:

- **label** (`str`) -- The class label identified by the model.
- **mask** (`PIL.Image`) -- A binary mask of the detected object as a Pil Image of shape (width, height) of
  the original image. Returns a mask filled with zeros if no object is found.
- **score** (*optional* `float`) -- Optionally, when the model is capable of estimating a confidence of the
  "object" described by the label and the mask.
#### transformers.KeypointMatchingPipeline[[transformers.KeypointMatchingPipeline]]

[Source](https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/pipelines/keypoint_matching.py#L69)

Keypoint matching pipeline using any `AutoModelForKeypointMatching`. This pipeline matches keypoints between two images.

__call__transformers.KeypointMatchingPipeline.__call__https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/pipelines/keypoint_matching.py#L98[{"name": "inputs", "val": ": list[collections.abc.Sequence[typing.Union[ForwardRef('Image.Image'), str]]] | collections.abc.Sequence[typing.Union[ForwardRef('Image.Image'), str]]"}, {"name": "threshold", "val": ": float = 0.0"}, {"name": "**kwargs", "val": ": typing.Any"}]- **inputs** (`str`, `list[str]`, `PIL.Image` or `list[PIL.Image]`) --
  The pipeline handles three types of images:

  - A string containing a http link pointing to an image
  - A string containing a local path to an image
  - An image loaded in PIL directly
The pipeline accepts either a single pair of images or a batch of image pairs, which must then be passed as a string.
  Images in a batch must all be in the same format: all as http links, all as local paths, or all as PIL
  images.

- **threshold** (`float`, *optional*, defaults to 0.0) --
  The threshold to use for keypoint matching. Keypoints matched with a lower matching score will be filtered out.
  A value of 0 means that all matched keypoints will be returned.

- **kwargs** --
  `timeout (`float`, *optional*, defaults to None)`
  The maximum time in seconds to wait for fetching images from the web. If None, no timeout is set and
  the call may block forever.0Union[list[Match], list[list[Match]]]A list of matches or a list if a single image pair is provided, or of lists of matches if a batch
of image pairs is provided. Each match is a dictionary containing the following keys:
- **keypoint_image_0** (`Keypoint`): The keypoint in the first image (x, y coordinates).
- **keypoint_image_1** (`Keypoint`): The keypoint in the second image (x, y coordinates).
- **score** (`float`): The matching score between the two keypoints.

Find matches between keypoints in two images.

**Parameters:**

inputs (`str`, `list[str]`, `PIL.Image` or `list[PIL.Image]`) : The pipeline handles three types of images:  - A string containing a http link pointing to an image - A string containing a local path to an image - An image loaded in PIL directly  The pipeline accepts either a single pair of images or a batch of image pairs, which must then be passed as a string. Images in a batch must all be in the same format: all as http links, all as local paths, or all as PIL images.

threshold (`float`, *optional*, defaults to 0.0) : The threshold to use for keypoint matching. Keypoints matched with a lower matching score will be filtered out. A value of 0 means that all matched keypoints will be returned.
kwargs : `timeout (`float`, *optional*, defaults to None)` The maximum time in seconds to wait for fetching images from the web. If None, no timeout is set and the call may block forever.

**Returns:**

`Union[list[Match], list[list[Match]]]`

A list of matches or a list if a single image pair is provided, or of lists of matches if a batch
of image pairs is provided. Each match is a dictionary containing the following keys:

- **keypoint_image_0** (`Keypoint`): The keypoint in the first image (x, y coordinates).
- **keypoint_image_1** (`Keypoint`): The keypoint in the second image (x, y coordinates).
- **score** (`float`): The matching score between the two keypoints.
#### transformers.ObjectDetectionPipeline[[transformers.ObjectDetectionPipeline]]

[Source](https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/pipelines/object_detection.py#L26)

Object detection pipeline using any `AutoModelForObjectDetection`. This pipeline predicts bounding boxes of objects
and their classes.

Example:

```python
>>> from transformers import pipeline

>>> detector = pipeline(model="facebook/detr-resnet-50")
>>> detector("https://huggingface.co/datasets/Narsil/image_dummy/raw/main/parrots.png")
[{'score': 0.997, 'label': 'bird', 'box': {'xmin': 69, 'ymin': 171, 'xmax': 396, 'ymax': 507}}, {'score': 0.999, 'label': 'bird', 'box': {'xmin': 398, 'ymin': 105, 'xmax': 767, 'ymax': 507}}]

>>> # x, y  are expressed relative to the top left hand corner.
```

Learn more about the basics of using a pipeline in the [pipeline tutorial](../pipeline_tutorial)
This object detection pipeline can currently be loaded from [pipeline()](/docs/transformers/v5.6.2/en/main_classes/pipelines#transformers.pipeline) using the following task identifier:
`"object-detection"`.

See the list of available models on [huggingface.co/models](https://huggingface.co/models?filter=object-detection).

__call__transformers.ObjectDetectionPipeline.__call__https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/pipelines/object_detection.py#L81[{"name": "*args", "val": ""}, {"name": "**kwargs", "val": ""}]- **inputs** (`str`, `list[str]`, `PIL.Image` or `list[PIL.Image]`) --
  The pipeline handles three types of images:

  - A string containing an HTTP(S) link pointing to an image
  - A string containing a local path to an image
  - An image loaded in PIL directly
The pipeline accepts either a single image or a batch of images. Images in a batch must all be in the
  same format: all as HTTP(S) links, all as local paths, or all as PIL images.
- **threshold** (`float`, *optional*, defaults to 0.5) --
  The probability necessary to make a prediction.
- **timeout** (`float`, *optional*, defaults to None) --
  The maximum time in seconds to wait for fetching images from the web. If None, no timeout is set and
  the call may block forever.0A list of dictionaries or a list of list of dictionaries containing the result. If the input is a single
image, will return a list of dictionaries, if the input is a list of several images, will return a list of
list of dictionaries corresponding to each image.

The dictionaries contain the following keys:

- **label** (`str`) -- The class label identified by the model.
- **score** (`float`) -- The score attributed by the model for that label.
- **box** (`list[dict[str, int]]`) -- The bounding box of detected object in image's original size.
Detect objects (bounding boxes & classes) in the image(s) passed as inputs.

**Parameters:**

model ([PreTrainedModel](/docs/transformers/v5.6.2/en/main_classes/model#transformers.PreTrainedModel)) : The model that will be used by the pipeline to make predictions. This needs to be a model inheriting from [PreTrainedModel](/docs/transformers/v5.6.2/en/main_classes/model#transformers.PreTrainedModel).

image_processor ([BaseImageProcessor](/docs/transformers/v5.6.2/en/main_classes/image_processor#transformers.BaseImageProcessor)) : The image processor that will be used by the pipeline to encode data for the model. This object inherits from [BaseImageProcessor](/docs/transformers/v5.6.2/en/main_classes/image_processor#transformers.BaseImageProcessor).

task (`str`, defaults to `""`) : A task-identifier for the pipeline.

num_workers (`int`, *optional*, defaults to 8) : When the pipeline will use *DataLoader* (when passing a dataset, on GPU for a Pytorch model), the number of workers to be used.
binary_output (`bool`, *optional*, defaults to `False`) : Flag indicating if the output the pipeline should happen in a serialized format (i.e., pickle) or as the raw output data e.g. text.

**Returns:**

A list of dictionaries or a list of list of dictionaries containing the result. If the input is a single
image, will return a list of dictionaries, if the input is a list of several images, will return a list of
list of dictionaries corresponding to each image.

The dictionaries contain the following keys:

- **label** (`str`) -- The class label identified by the model.
- **score** (`float`) -- The score attributed by the model for that label.
- **box** (`list[dict[str, int]]`) -- The bounding box of detected object in image's original size.
#### transformers.VideoClassificationPipeline[[transformers.VideoClassificationPipeline]]

[Source](https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/pipelines/video_classification.py#L41)

Video classification pipeline using any `AutoModelForVideoClassification`. This pipeline predicts the class of a
video.

This video classification pipeline can currently be loaded from [pipeline()](/docs/transformers/v5.6.2/en/main_classes/pipelines#transformers.pipeline) using the following task identifier:
`"video-classification"`.

See the list of available models on
[huggingface.co/models](https://huggingface.co/models?filter=video-classification).

__call__transformers.VideoClassificationPipeline.__call__https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/pipelines/video_classification.py#L90[{"name": "inputs", "val": ": str | list[str] | None"}, {"name": "**kwargs", "val": ""}]- **inputs** (`str`, `list[str]`) --
  The pipeline handles three types of videos:
The pipeline accepts either a single video or a batch of videos, which must then be passed as a string.
  Videos in a batch must all be in the same format: all as http links or all as local paths.
- **top_k** (`int`, *optional*, defaults to 5) --
  The number of top labels that will be returned by the pipeline. If the provided number is higher than
  the number of labels available in the model configuration, it will default to the number of labels.
- **num_frames** (`int`, *optional*, defaults to `self.model.config.num_frames`) --
  The number of frames sampled from the video to run the classification on. If not provided, will default
  to the number of frames specified in the model configuration.
- **frame_sampling_rate** (`int`, *optional*, defaults to 1) --
  The sampling rate used to select frames from the video. If not provided, will default to 1, i.e. every
  frame will be used.
- **function_to_apply(`str`,** *optional*, defaults to "softmax") --
  The function to apply to the model output. By default, the pipeline will apply the softmax function to
  the output of the model. Valid options: ["softmax", "sigmoid", "none"]. Note that passing Python's
  built-in `None` will default to "softmax", so you need to pass the string "none" to disable any
  post-processing.0A list of dictionaries or a list of list of dictionaries containing result. If the input is a single video,
will return a list of `top_k` dictionaries, if the input is a list of several videos, will return a list of list of
`top_k` dictionaries corresponding to the videos.
The dictionaries contain the following keys:

- **label** (`str`) -- The label identified by the model.
- **score** (`int`) -- The score attributed by the model for that label.

Assign labels to the video(s) passed as inputs.

**Parameters:**

model ([PreTrainedModel](/docs/transformers/v5.6.2/en/main_classes/model#transformers.PreTrainedModel)) : The model that will be used by the pipeline to make predictions. This needs to be a model inheriting from [PreTrainedModel](/docs/transformers/v5.6.2/en/main_classes/model#transformers.PreTrainedModel).

image_processor ([BaseImageProcessor](/docs/transformers/v5.6.2/en/main_classes/image_processor#transformers.BaseImageProcessor)) : The image processor that will be used by the pipeline to encode data for the model. This object inherits from [BaseImageProcessor](/docs/transformers/v5.6.2/en/main_classes/image_processor#transformers.BaseImageProcessor).

task (`str`, defaults to `""`) : A task-identifier for the pipeline.
dtype (`str` or `torch.dtype`, *optional*) : Sent directly as `model_kwargs` (just a simpler shortcut) to use the available precision for this model (`torch.float16`, `torch.bfloat16`, ... or `"auto"`)

binary_output (`bool`, *optional*, defaults to `False`) : Flag indicating if the output the pipeline should happen in a serialized format (i.e., pickle) or as the raw output data e.g. text.

**Returns:**

A list of dictionaries or a list of list of dictionaries containing result. If the input is a single video,
will return a list of `top_k` dictionaries, if the input is a list of several videos, will return a list of list of
`top_k` dictionaries corresponding to the videos.

The dictionaries contain the following keys:

- **label** (`str`) -- The label identified by the model.
- **score** (`int`) -- The score attributed by the model for that label.
#### transformers.ZeroShotImageClassificationPipeline[[transformers.ZeroShotImageClassificationPipeline]]

[Source](https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/pipelines/zero_shot_image_classification.py#L29)

Zero shot image classification pipeline using `CLIPModel`. This pipeline predicts the class of an image when you
provide an image and a set of `candidate_labels`.

Example:
```python
>>> from transformers import pipeline

>>> classifier = pipeline(model="google/siglip-so400m-patch14-384")
>>> classifier(
...     "https://huggingface.co/datasets/Narsil/image_dummy/raw/main/parrots.png",
...     candidate_labels=["animals", "humans", "landscape"],
... )
[{'score': 0.965, 'label': 'animals'}, {'score': 0.03, 'label': 'humans'}, {'score': 0.005, 'label': 'landscape'}]

>>> classifier(
...     "https://huggingface.co/datasets/Narsil/image_dummy/raw/main/parrots.png",
...     candidate_labels=["black and white", "photorealist", "painting"],
... )
[{'score': 0.996, 'label': 'black and white'}, {'score': 0.003, 'label': 'photorealist'}, {'score': 0.0, 'label': 'painting'}]
```

Learn more about the basics of using a pipeline in the [pipeline tutorial](../pipeline_tutorial)
This image classification pipeline can currently be loaded from [pipeline()](/docs/transformers/v5.6.2/en/main_classes/pipelines#transformers.pipeline) using the following task identifier:
`"zero-shot-image-classification"`.

See the list of available models on
[huggingface.co/models](https://huggingface.co/models?filter=zero-shot-image-classification).

__call__transformers.ZeroShotImageClassificationPipeline.__call__https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/pipelines/zero_shot_image_classification.py#L83[{"name": "image", "val": ": typing.Union[str, list[str], ForwardRef('Image.Image'), list['Image.Image']]"}, {"name": "candidate_labels", "val": ": list"}, {"name": "**kwargs", "val": ": typing.Any"}]- **image** (`str`, `list[str]`, `PIL.Image` or `list[PIL.Image]`) --
  The pipeline handles three types of images:

  - A string containing a http link pointing to an image
  - A string containing a local path to an image
  - An image loaded in PIL directly
- **candidate_labels** (`list[str]`) --
  The candidate labels for this image. They will be formatted using *hypothesis_template*.

- **hypothesis_template** (`str`, *optional*, defaults to `"This is a photo of {}"`) --
  The format used in conjunction with *candidate_labels* to attempt the image classification by
  replacing the placeholder with the candidate_labels. Pass "{}" if *candidate_labels* are
  already formatted.

- **timeout** (`float`, *optional*, defaults to None) --
  The maximum time in seconds to wait for fetching images from the web. If None, no timeout is set and
  the call may block forever.0A list of dictionaries containing one entry per proposed label. Each dictionary contains the
following keys:
- **label** (`str`) -- One of the suggested *candidate_labels*.
- **score** (`float`) -- The score attributed by the model to that label. It is a value between
  0 and 1, computed as the `softmax` of `logits_per_image`.

Assign labels to the image(s) passed as inputs.

**Parameters:**
model ([PreTrainedModel](/docs/transformers/v5.6.2/en/main_classes/model#transformers.PreTrainedModel)) : The model that will be used by the pipeline to make predictions. This needs to be a model inheriting from [PreTrainedModel](/docs/transformers/v5.6.2/en/main_classes/model#transformers.PreTrainedModel).

image_processor ([BaseImageProcessor](/docs/transformers/v5.6.2/en/main_classes/image_processor#transformers.BaseImageProcessor)) : The image processor that will be used by the pipeline to encode data for the model. This object inherits from [BaseImageProcessor](/docs/transformers/v5.6.2/en/main_classes/image_processor#transformers.BaseImageProcessor).

task (`str`, defaults to `""`) : A task-identifier for the pipeline.

num_workers (`int`, *optional*, defaults to 8) : When the pipeline will use *DataLoader* (when passing a dataset, on GPU for a Pytorch model), the number of workers to be used.
binary_output (`bool`, *optional*, defaults to `False`) : Flag indicating if the output the pipeline should happen in a serialized format (i.e., pickle) or as the raw output data e.g. text.

**Returns:**

A list of dictionaries containing one entry per proposed label. Each dictionary contains the
following keys:
- **label** (`str`) -- One of the suggested *candidate_labels*.
- **score** (`float`) -- The score attributed by the model to that label. It is a value between
  0 and 1, computed as the `softmax` of `logits_per_image`.
#### transformers.ZeroShotObjectDetectionPipeline[[transformers.ZeroShotObjectDetectionPipeline]]

[Source](https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/pipelines/zero_shot_object_detection.py#L23)

Zero shot object detection pipeline using `OwlViTForObjectDetection`. This pipeline predicts bounding boxes of
objects when you provide an image and a set of `candidate_labels`.

Example:
```python
>>> from transformers import pipeline

>>> detector = pipeline(model="google/owlvit-base-patch32", task="zero-shot-object-detection")
>>> detector(
...     "http://images.cocodataset.org/val2017/000000039769.jpg",
...     candidate_labels=["cat", "couch"],
... )
[{'score': 0.287, 'label': 'cat', 'box': {'xmin': 324, 'ymin': 20, 'xmax': 640, 'ymax': 373}}, {'score': 0.254, 'label': 'cat', 'box': {'xmin': 1, 'ymin': 55, 'xmax': 315, 'ymax': 472}}, {'score': 0.121, 'label': 'couch', 'box': {'xmin': 4, 'ymin': 0, 'xmax': 642, 'ymax': 476}}]

>>> detector(
...     "https://huggingface.co/datasets/Narsil/image_dummy/raw/main/parrots.png",
...     candidate_labels=["head", "bird"],
... )
[{'score': 0.119, 'label': 'bird', 'box': {'xmin': 71, 'ymin': 170, 'xmax': 410, 'ymax': 508}}]
```

Learn more about the basics of using a pipeline in the [pipeline tutorial](../pipeline_tutorial)
This object detection pipeline can currently be loaded from [pipeline()](/docs/transformers/v5.6.2/en/main_classes/pipelines#transformers.pipeline) using the following task identifier:
`"zero-shot-object-detection"`.

See the list of available models on
[huggingface.co/models](https://huggingface.co/models?filter=zero-shot-object-detection).

__call__transformers.ZeroShotObjectDetectionPipeline.__call__https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/pipelines/zero_shot_object_detection.py#L75[{"name": "image", "val": ": typing.Union[str, ForwardRef('Image.Image'), list[dict[str, typing.Any]]]"}, {"name": "candidate_labels", "val": ": str | list[str] | None = None"}, {"name": "**kwargs", "val": ": typing.Any"}]- **image** (`str`, `PIL.Image` or `list[dict[str, Any]]`) --
  The pipeline handles three types of images:

  - A string containing an http url pointing to an image
  - A string containing a local path to an image
  - An image loaded in PIL directly
```python
>>> from transformers import pipeline

>>> detector = pipeline(model="google/owlvit-base-patch32", task="zero-shot-object-detection")
>>> detector(
...     [
...         {
...             "image": "http://images.cocodataset.org/val2017/000000039769.jpg",
...             "candidate_labels": ["cat", "couch"],
...         },
...         {
...             "image": "http://images.cocodataset.org/val2017/000000039769.jpg",
...             "candidate_labels": ["cat", "couch"],
...         },
...     ]
... )
[[{'score': 0.287, 'label': 'cat', 'box': {'xmin': 324, 'ymin': 20, 'xmax': 640, 'ymax': 373}}, {'score': 0.25, 'label': 'cat', 'box': {'xmin': 1, 'ymin': 55, 'xmax': 315, 'ymax': 472}}, {'score': 0.121, 'label': 'couch', 'box': {'xmin': 4, 'ymin': 0, 'xmax': 642, 'ymax': 476}}], [{'score': 0.287, 'label': 'cat', 'box': {'xmin': 324, 'ymin': 20, 'xmax': 640, 'ymax': 373}}, {'score': 0.254, 'label': 'cat', 'box': {'xmin': 1, 'ymin': 55, 'xmax': 315, 'ymax': 472}}, {'score': 0.121, 'label': 'couch', 'box': {'xmin': 4, 'ymin': 0, 'xmax': 642, 'ymax': 476}}]]
```