philipp-zettl/vrom-hf-docs huggingface dataset:
__call__transformers.MaskGenerationPipeline.__call__https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/pipelines/mask_generation.py#L143[{"name": "image", "val": ": typing.Union[str, ForwardRef('Image.Image'), list[str], list['Image.Image']]"}, {"name": "*args", "val": ": typing.Any"}, {"name": "**kwargs", "val": ": typing.Any"}]- **image** (`str`, `List[str]`, `PIL.Image` or `List[PIL.Image]`) --
  Image or list of images.
- **mask_threshold** (`float`, *optional*, defaults to 0.0) --
  Threshold to use when turning the predicted masks into binary values.
- **pred_iou_thresh** (`float`, *optional*, defaults to 0.88) --
  A filtering threshold in `[0,1]` applied on the model's predicted mask quality.
- **stability_score_thresh** (`float`, *optional*, defaults to 0.95) --
  A filtering threshold in `[0,1]`, using the stability of the mask under changes to the cutoff used to
  binarize the model's mask predictions.
- **stability_score_offset** (`int`, *optional*, defaults to 1) --
  The amount to shift the cutoff when calculated the stability score.
- **crops_nms_thresh** (`float`, *optional*, defaults to 0.7) --
  The box IoU cutoff used by non-maximal suppression to filter duplicate masks.
- **crops_n_layers** (`int`, *optional*, defaults to 0) --
  If `crops_n_layers>0`, mask prediction will be run again on crops of the image. Sets the number of
  layers to run, where each layer has 2**i_layer number of image crops.
- **crop_overlap_ratio** (`float`, *optional*, defaults to `512 / 1500`) --
  Sets the degree to which crops overlap. In the first crop layer, crops will overlap by this fraction of
  the image length. Later layers with more crops scale down this overlap.
- **crop_n_points_downscale_factor** (`int`, *optional*, defaults to `1`) --
  The number of points-per-side sampled in layer n is scaled down by crop_n_points_downscale_factor**n.
- **timeout** (`float`, *optional*, defaults to None) --
  The maximum time in seconds to wait for fetching images from the web. If None, no timeout is set and
  the call may block forever.0`Dict`A dictionary with the following keys:
- **mask** (`PIL.Image`) -- A binary mask of the detected object as a PIL Image of shape `(width,
  height)` of the original image. Returns a mask filled with zeros if no object is found.
- **score** (*optional* `float`) -- Optionally, when the model is capable of estimating a confidence of
  the "object" described by the label and the mask.
Generates binary segmentation masks

**Parameters:**

model ([PreTrainedModel](/docs/transformers/v5.6.2/en/main_classes/model#transformers.PreTrainedModel)) : The model that will be used by the pipeline to make predictions. This needs to be a model inheriting from [PreTrainedModel](/docs/transformers/v5.6.2/en/main_classes/model#transformers.PreTrainedModel).

image_processor ([BaseImageProcessor](/docs/transformers/v5.6.2/en/main_classes/image_processor#transformers.BaseImageProcessor)) : The image processor that will be used by the pipeline to encode data for the model. This object inherits from [BaseImageProcessor](/docs/transformers/v5.6.2/en/main_classes/image_processor#transformers.BaseImageProcessor).

task (`str`, defaults to `""`) : A task-identifier for the pipeline.

num_workers (`int`, *optional*, defaults to 8) : When the pipeline will use *DataLoader* (when passing a dataset, on GPU for a Pytorch model), the number of workers to be used.
binary_output (`bool`, *optional*, defaults to `False`) : Flag indicating if the output the pipeline should happen in a serialized format (i.e., pickle) or as the raw output data e.g. text.

points_per_batch (*optional*, int, default to 64) : Sets the number of points run simultaneously by the model. Higher numbers may be faster but use more GPU memory.

output_bboxes_mask (`bool`, *optional*, default to `False`) : Whether or not to output the bounding box predictions.

output_rle_masks (`bool`, *optional*, default to `False`) : Whether or not to output the masks in `RLE` format

**Returns:**

``Dict``

A dictionary with the following keys:
- **mask** (`PIL.Image`) -- A binary mask of the detected object as a PIL Image of shape `(width,
  height)` of the original image. Returns a mask filled with zeros if no object is found.
- **score** (*optional* `float`) -- Optionally, when the model is capable of estimating a confidence of
  the "object" described by the label and the mask.
#### transformers.Pipeline[[transformers.Pipeline]]

[Source](https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/pipelines/base.py#L737)

The Pipeline class is the class from which all pipelines inherit. Refer to this class for methods shared across
different pipelines.

Base class implementing pipelined operations. Pipeline workflow is defined as a sequence of the following
operations:

Input -> Tokenization -> Model Inference -> Post-Processing (task dependent) -> Output

Pipeline supports running on CPU or GPU through the device argument (see below).

Some pipeline, like for instance [FeatureExtractionPipeline](/docs/transformers/v5.6.2/en/main_classes/pipelines#transformers.FeatureExtractionPipeline) (`'feature-extraction'`) output large tensor object
as nested-lists. In order to avoid dumping such large structure as textual data we provide the `binary_output`
constructor argument. If set to `True`, the output will be stored in the pickle format.
check_model_typetransformers.Pipeline.check_model_typehttps://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/pipelines/base.py#L1079[{"name": "supported_models", "val": ": list[str] | dict"}]- **supported_models** (`list[str]` or `dict`) --
  The list of models supported by the pipeline, or a dictionary with model class values.0

Check if the model class is in supported by the pipeline.

**Parameters:**

model ([PreTrainedModel](/docs/transformers/v5.6.2/en/main_classes/model#transformers.PreTrainedModel)) : The model that will be used by the pipeline to make predictions. This needs to be a model inheriting from [PreTrainedModel](/docs/transformers/v5.6.2/en/main_classes/model#transformers.PreTrainedModel).
tokenizer ([PreTrainedTokenizer](/docs/transformers/v5.6.2/en/main_classes/tokenizer#transformers.PythonBackend)) : The tokenizer that will be used by the pipeline to encode data for the model. This object inherits from [PreTrainedTokenizer](/docs/transformers/v5.6.2/en/main_classes/tokenizer#transformers.PythonBackend).

feature_extractor ([SequenceFeatureExtractor](/docs/transformers/v5.6.2/en/main_classes/feature_extractor#transformers.SequenceFeatureExtractor)) : The feature extractor that will be used by the pipeline to encode data for the model. This object inherits from [SequenceFeatureExtractor](/docs/transformers/v5.6.2/en/main_classes/feature_extractor#transformers.SequenceFeatureExtractor).
image_processor ([BaseImageProcessor](/docs/transformers/v5.6.2/en/main_classes/image_processor#transformers.BaseImageProcessor)) : The image processor that will be used by the pipeline to encode data for the model. This object inherits from [BaseImageProcessor](/docs/transformers/v5.6.2/en/main_classes/image_processor#transformers.BaseImageProcessor).

processor ([ProcessorMixin](/docs/transformers/v5.6.2/en/main_classes/processors#transformers.ProcessorMixin)) : The processor that will be used by the pipeline to encode data for the model. This object inherits from [ProcessorMixin](/docs/transformers/v5.6.2/en/main_classes/processors#transformers.ProcessorMixin). Processor is a composite object that might contain `tokenizer`, `feature_extractor`, and `image_processor`.

task (`str`, defaults to `""`) : A task-identifier for the pipeline.
dtype (`str` or `torch.dtype`, *optional*) : Sent directly as `model_kwargs` (just a simpler shortcut) to use the available precision for this model (`torch.float16`, `torch.bfloat16`, ... or `"auto"`)

binary_output (`bool`, *optional*, defaults to `False`) : Flag indicating if the output the pipeline should happen in a serialized format (i.e., pickle) or as the raw output data e.g. text.
#### ensure_tensor_on_device[[transformers.Pipeline.ensure_tensor_on_device]]

[Source](https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/pipelines/base.py#L1047)

Ensure PyTorch tensors are on the specified device.

**Parameters:**

inputs (keyword arguments that should be `torch.Tensor`, the rest is ignored) : The tensors to place on `self.device`.

Recursive on lists **only**. --

**Returns:**

``dict[str, torch.Tensor]``

The same as `inputs` but on the proper device.
#### postprocess[[transformers.Pipeline.postprocess]]

[Source](https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/pipelines/base.py#L1146)

Postprocess will receive the raw outputs of the `_forward` method, generally tensors, and reformat them into
something more friendly. Generally it will output a list or a dict or results (containing just strings and
numbers).
#### preprocess[[transformers.Pipeline.preprocess]]

[Source](https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/pipelines/base.py#L1125)

Preprocess will take the `input_` of a specific pipeline and return a dictionary of everything necessary for
`_forward` to run properly. It should contain at least one tensor, but might have arbitrary other items.
#### push_to_hub[[transformers.Pipeline.push_to_hub]]

[Source](https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/utils/hub.py#L720)

Upload the pipeline file to the 🤗 Model Hub.

Examples:

```python
from transformers import pipeline

pipe = pipeline("google-bert/bert-base-cased")
# Push the pipe to an organization with the name "my-finetuned-bert".
pipe.push_to_hub("huggingface/my-finetuned-bert")
```

**Parameters:**

repo_id (`str`) : The name of the repository you want to push your pipe to. It should contain your organization name when pushing to a given organization.

commit_message (`str`, *optional*) : Message to commit while pushing. Will default to `"Upload pipe"`.

commit_description (`str`, *optional*) : The description of the commit that will be created

private (`bool`, *optional*) : Whether to make the repo private. If `None` (default), the repo will be public unless the organization's default is private. This value is ignored if the repo already exists.

token (`bool` or `str`, *optional*) : The token to use as HTTP bearer authorization for remote files. If `True` (default), will use the token generated when running `hf auth login` (stored in `~/.huggingface`).

revision (`str`, *optional*) : Branch to push the uploaded files to.
#### save_pretrained[[transformers.Pipeline.save_pretrained]]

[Source](https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/pipelines/base.py#L942)

Save the pipeline's model and tokenizer.

**Parameters:**

save_directory (`str` or `os.PathLike`) : A path to the directory where to saved. It will be created if it doesn't exist.

kwargs (`dict[str, Any]`, *optional*) : Additional key word arguments passed along to the [push_to_hub()](/docs/transformers/v5.6.2/en/main_classes/model#transformers.utils.PushToHubMixin.push_to_hub) method.
# Trainer

The [Trainer](/docs/transformers/v5.6.2/en/main_classes/trainer#transformers.Trainer) class provides an API for feature-complete training in PyTorch, and it supports distributed training on multiple GPUs/TPUs, mixed precision for [NVIDIA GPUs](https://nvidia.github.io/apex/), [AMD GPUs](https://rocm.docs.amd.com/en/latest/rocm.html), and [`torch.amp`](https://pytorch.org/docs/stable/amp.html) for PyTorch. [Trainer](/docs/transformers/v5.6.2/en/main_classes/trainer#transformers.Trainer) goes hand-in-hand with the [TrainingArguments](/docs/transformers/v5.6.2/en/main_classes/trainer#transformers.TrainingArguments) class, which offers a wide range of options to customize how a model is trained. Together, these two classes provide a complete training API.
[Seq2SeqTrainer](/docs/transformers/v5.6.2/en/main_classes/trainer#transformers.Seq2SeqTrainer) and [Seq2SeqTrainingArguments](/docs/transformers/v5.6.2/en/main_classes/trainer#transformers.Seq2SeqTrainingArguments) inherit from the [Trainer](/docs/transformers/v5.6.2/en/main_classes/trainer#transformers.Trainer) and [TrainingArguments](/docs/transformers/v5.6.2/en/main_classes/trainer#transformers.TrainingArguments) classes and they're adapted for training models for sequence-to-sequence tasks such as summarization or translation.

The [Trainer](/docs/transformers/v5.6.2/en/main_classes/trainer#transformers.Trainer) class is optimized for 🤗 Transformers models and can have surprising behaviors
when used with other models. When using it with your own model, make sure:
- your model always return tuples or subclasses of [ModelOutput](/docs/transformers/v5.6.2/en/main_classes/output#transformers.utils.ModelOutput)
- your model can compute the loss if a `labels` argument is provided and that loss is returned as the first
  element of the tuple (if your model returns tuples)
- your model can accept multiple label arguments (use `label_names` in [TrainingArguments](/docs/transformers/v5.6.2/en/main_classes/trainer#transformers.TrainingArguments) to indicate their name to the [Trainer](/docs/transformers/v5.6.2/en/main_classes/trainer#transformers.Trainer)) but none of them should be named `"label"`
- **model** -- Always points to the core model. If using a transformers model, it will be a [PreTrainedModel](/docs/transformers/v5.6.2/en/main_classes/model#transformers.PreTrainedModel)
  subclass.
- **model_wrapped** -- Always points to the most external model in case one or more other modules wrap the
  original model. This is the model that should be used for the forward pass. For example, under `DeepSpeed`,
  the inner model is wrapped in `DeepSpeed` and then again in `torch.nn.DistributedDataParallel`. If the inner
  model hasn't been wrapped, then `self.model_wrapped` is the same as `self.model`.
- **is_model_parallel** -- Whether or not a model has been switched to a model parallel mode (different from
  data parallelism, this means some of the model layers are split on different GPUs).
- **place_model_on_device** -- Whether or not to automatically place the model on the device. Defaults to
  `True` unless model parallel, DeepSpeed, FSDP, full fp16/bf16 eval, or SageMaker MP is active. Can be
  overridden by subclassing `TrainingArguments` and overriding the `place_model_on_device` property.
- **is_in_train** -- Whether or not a model is currently running `train` (e.g. when `evaluate` is called while
  in `train`)
add_callbacktransformers.Trainer.add_callbackhttps://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/trainer.py#L4350[{"name": "callback", "val": ": type[transformers.trainer_callback.TrainerCallback] | transformers.trainer_callback.TrainerCallback"}]- **callback** (`type` or [`~transformers.TrainerCallback]`) --
  A [TrainerCallback](/docs/transformers/v5.6.2/en/main_classes/callback#transformers.TrainerCallback) class or an instance of a [TrainerCallback](/docs/transformers/v5.6.2/en/main_classes/callback#transformers.TrainerCallback). In the
  first case, will instantiate a member of that class.0

Add a callback to the current list of [TrainerCallback](/docs/transformers/v5.6.2/en/main_classes/callback#transformers.TrainerCallback).

**Parameters:**
model ([PreTrainedModel](/docs/transformers/v5.6.2/en/main_classes/model#transformers.PreTrainedModel) or `torch.nn.Module`, *optional*) : The model to train, evaluate or use for predictions. If not provided, a `model_init` must be passed.    [Trainer](/docs/transformers/v5.6.2/en/main_classes/trainer#transformers.Trainer) is optimized to work with the [PreTrainedModel](/docs/transformers/v5.6.2/en/main_classes/model#transformers.PreTrainedModel) provided by the library. You can still use your own models defined as `torch.nn.Module` as long as they work the same way as the 🤗 Transformers models.   

args ([TrainingArguments](/docs/transformers/v5.6.2/en/main_classes/trainer#transformers.TrainingArguments), *optional*) : The arguments to tweak for training. Will default to a basic instance of [TrainingArguments](/docs/transformers/v5.6.2/en/main_classes/trainer#transformers.TrainingArguments) with the `output_dir` set to a directory named *tmp_trainer* in the current directory if not provided.
data_collator (`DataCollator`, *optional*) : The function to use to form a batch from a list of elements of `train_dataset` or `eval_dataset`. Will default to [default_data_collator()](/docs/transformers/v5.6.2/en/main_classes/data_collator#transformers.default_data_collator) if no `processing_class` is provided, an instance of [DataCollatorWithPadding](/docs/transformers/v5.6.2/en/main_classes/data_collator#transformers.DataCollatorWithPadding) otherwise if the processing_class is a feature extractor or tokenizer.
train_dataset (`torch.utils.data.Dataset` | `torch.utils.data.IterableDataset` | `datasets.Dataset`, *optional*) : The dataset to use for training. If it is a [Dataset](https://huggingface.co/docs/datasets/v4.8.4/en/package_reference/main_classes#datasets.Dataset), columns not accepted by the `model.forward()` method are automatically removed.  Note that if it's a `torch.utils.data.IterableDataset` with some randomization and you are training in a distributed fashion, your iterable dataset should either use a internal attribute `generator` that is a `torch.Generator` for the randomization that must be identical on all processes (and the Trainer will manually set the seed of this `generator` at each epoch) or have a `set_epoch()` method that internally sets the seed of the RNGs used.
eval_dataset (`torch.utils.data.Dataset` | dict[str, `torch.utils.data.Dataset`] | `datasets.Dataset`, *optional*) : The dataset to use for evaluation. If it is a [Dataset](https://huggingface.co/docs/datasets/v4.8.4/en/package_reference/main_classes#datasets.Dataset), columns not accepted by the `model.forward()` method are automatically removed. If it is a dictionary, it will evaluate on each dataset prepending the dictionary key to the metric name.

processing_class (`PreTrainedTokenizerBase` or `BaseImageProcessor` or `FeatureExtractionMixin` or `ProcessorMixin`, *optional*) : Processing class used to process the data. If provided, will be used to automatically process the inputs for the model, and it will be saved along the model to make it easier to rerun an interrupted training or reuse the fine-tuned model.
model_init (`Callable[[], PreTrainedModel]`, *optional*) : A function that instantiates the model to be used. If provided, each call to [train()](/docs/transformers/v5.6.2/en/main_classes/trainer#transformers.Trainer.train) will start from a new instance of the model as given by this function.  The function may have zero argument, or a single one containing the optuna/Ray Tune trial object, to be able to choose different architectures according to hyperparameters (such as layer count, sizes of inner layers, dropout probabilities etc).

compute_loss_func (`Callable`, *optional*) : A function that accepts the raw model outputs, labels, and the number of items in the entire accumulated batch (batch_size * gradient_accumulation_steps) and returns the loss. For example, see the default [loss function](https://github.com/huggingface/transformers/blob/052e652d6d53c2b26ffde87e039b723949a53493/src/transformers/trainer.py#L3618) used by [Trainer](/docs/transformers/v5.6.2/en/main_classes/trainer#transformers.Trainer).
compute_metrics (`Callable[[EvalPrediction], Dict]`, *optional*) : The function that will be used to compute metrics at evaluation. Must take a [EvalPrediction](/docs/transformers/v5.6.2/en/internal/trainer_utils#transformers.EvalPrediction) and return a dictionary string to metric values. *Note* When passing TrainingArgs with `batch_eval_metrics` set to `True`, your compute_metrics function must take a boolean `compute_result` argument. This will be triggered after the last eval batch to signal that the function needs to calculate and return the global summary statistics rather than accumulating the batch-level statistics
callbacks (List of [TrainerCallback](/docs/transformers/v5.6.2/en/main_classes/callback#transformers.TrainerCallback), *optional*) : A list of callbacks to customize the training loop. Will add those to the list of default callbacks detailed in [here](callback).  If you want to remove one of the default callbacks used, use the [Trainer.remove_callback()](/docs/transformers/v5.6.2/en/main_classes/trainer#transformers.Trainer.remove_callback) method.

optimizers (`tuple[torch.optim.Optimizer, torch.optim.lr_scheduler.LambdaLR]`, *optional*, defaults to `(None, None)`) : A tuple containing the optimizer and the scheduler to use. Will default to an instance of `AdamW` on your model and a scheduler given by [get_linear_schedule_with_warmup()](/docs/transformers/v5.6.2/en/main_classes/optimizer_schedules#transformers.get_linear_schedule_with_warmup) controlled by `args`.
optimizer_cls_and_kwargs (`tuple[Type[torch.optim.Optimizer], dict[str, Any]]`, *optional*) : A tuple containing the optimizer class and keyword arguments to use. Overrides `optim` and `optim_args` in `args`. Incompatible with the `optimizers` argument.  Unlike `optimizers`, this argument avoids the need to place model parameters on the correct devices before initializing the Trainer.

preprocess_logits_for_metrics (`Callable[[torch.Tensor, torch.Tensor], torch.Tensor]`, *optional*) : A function that preprocess the logits right before caching them at each evaluation step. Must take two tensors, the logits and the labels, and return the logits once processed as desired. The modifications made by this function will be reflected in the predictions received by `compute_metrics`.  Note that the labels (second parameter) will be `None` if the dataset does not have them.
#### autocast_smart_context_manager[[transformers.Trainer.autocast_smart_context_manager]]

[Source](https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/trainer.py#L2037)

A helper wrapper that creates an appropriate context manager for `autocast` while feeding it the desired
arguments, depending on the situation. We rely on accelerate for autocast, hence we do nothing here.
#### compute_loss[[transformers.Trainer.compute_loss]]

[Source](https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/trainer.py#L1941)

How the loss is computed by Trainer. By default, all models return the loss in the first element.

Subclass and override for custom behavior. If you are not using `num_items_in_batch` when computing your loss,
make sure to overwrite `self.model_accepts_loss_kwargs` to `False`. Otherwise, the loss calculation might be slightly inaccurate when performing gradient accumulation.

**Parameters:**

model (`nn.Module`) : The model to compute the loss for.

inputs (`dict[str, torch.Tensor | Any]`) : The input data for the model.

return_outputs (`bool`, *optional*, defaults to `False`) : Whether to return the model outputs along with the loss.

num_items_in_batch (Optional[torch.Tensor], *optional*) : The number of items in the batch. If not passed, the loss is computed using the default batch size reduction logic.

**Returns:**
#### create_model_card[[transformers.Trainer.create_model_card]]

[Source](https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/trainer.py#L3925)

Creates a draft of a model card using the information available to the `Trainer`.

**Parameters:**

language (`str`, *optional*) : The language of the model (if applicable)

license (`str`, *optional*) : The license of the model. Will default to the license of the pretrained model used, if the original model given to the `Trainer` comes from a repo on the Hub.

tags (`str` or `list[str]`, *optional*) : Some tags to be included in the metadata of the model card.

model_name (`str`, *optional*) : The name of the model.

finetuned_from (`str`, *optional*) : The name of the model used to fine-tune this one (if applicable). Will default to the name of the repo of the original model given to the `Trainer` (if it comes from the Hub).