philipp-zettl/vrom-hf-docs huggingface dataset:
#### train[[transformers.Trainer.train]]

[Source](https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/trainer.py#L1325)

Main training entry point.

**Parameters:**

resume_from_checkpoint (`str` or `bool`, *optional*) : If a `str`, local path to a saved checkpoint as saved by a previous instance of [Trainer](/docs/transformers/v5.6.2/en/main_classes/trainer#transformers.Trainer). If a `bool` and equals `True`, load the last checkpoint in *args.output_dir* as saved by a previous instance of [Trainer](/docs/transformers/v5.6.2/en/main_classes/trainer#transformers.Trainer). If present, training will resume from the model/optimizer/scheduler states loaded here.

trial (`optuna.Trial` or `dict[str, Any]`, *optional*) : The trial run or the hyperparameter dictionary for hyperparameter search.

ignore_keys_for_eval (`list[str]`, *optional*) : A list of keys in the output of your model (if it is a dictionary) that should be ignored when gathering predictions for evaluation during the training.
#### training_step[[transformers.Trainer.training_step]]

[Source](https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/trainer.py#L1870)

Perform a training step on a batch of inputs.

Subclass and override to inject custom behavior.

**Parameters:**

model (`nn.Module`) : The model to train.

inputs (`dict[str, torch.Tensor | Any]`) : The inputs and targets of the model.  The dictionary will be unpacked before being fed to the model. Most models expect the targets under the argument `labels`. Check your model's documentation for all accepted arguments.

**Returns:**

``torch.Tensor``

The tensor with training loss on this batch.
evaluatetransformers.Seq2SeqTrainer.evaluatehttps://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/trainer_seq2seq.py#L139[{"name": "eval_dataset", "val": ": torch.utils.data.dataset.Dataset | None = None"}, {"name": "ignore_keys", "val": ": list[str] | None = None"}, {"name": "metric_key_prefix", "val": ": str = 'eval'"}, {"name": "**gen_kwargs", "val": ""}]- **eval_dataset** (`Dataset`, *optional*) --
  Pass a dataset if you wish to override `self.eval_dataset`. If it is an [Dataset](https://huggingface.co/docs/datasets/v4.8.4/en/package_reference/main_classes#datasets.Dataset), columns
  not accepted by the `model.forward()` method are automatically removed. It must implement the `__len__`
  method.
- **ignore_keys** (`list[str]`, *optional*) --
  A list of keys in the output of your model (if it is a dictionary) that should be ignored when
  gathering predictions.
- **metric_key_prefix** (`str`, *optional*, defaults to `"eval"`) --
  An optional prefix to be used as the metrics key prefix. For example the metrics "bleu" will be named
  "eval_bleu" if the prefix is `"eval"` (default)
- **max_length** (`int`, *optional*) --
  The maximum target length to use when predicting with the generate method.
- **num_beams** (`int`, *optional*) --
  Number of beams for beam search that will be used when predicting with the generate method. 1 means no
  beam search.
- **gen_kwargs** --
  Additional `generate` specific kwargs.0A dictionary containing the evaluation loss and the potential metrics computed from the predictions. The
dictionary also contains the epoch number which comes from the training state.
Run evaluation and returns metrics.

The calling script will be responsible for providing a method to compute metrics, as they are task-dependent
(pass it to the init `compute_metrics` argument).

You can also subclass and override this method to inject custom behavior.

**Parameters:**

eval_dataset (`Dataset`, *optional*) : Pass a dataset if you wish to override `self.eval_dataset`. If it is an [Dataset](https://huggingface.co/docs/datasets/v4.8.4/en/package_reference/main_classes#datasets.Dataset), columns not accepted by the `model.forward()` method are automatically removed. It must implement the `__len__` method.

ignore_keys (`list[str]`, *optional*) : A list of keys in the output of your model (if it is a dictionary) that should be ignored when gathering predictions.

metric_key_prefix (`str`, *optional*, defaults to `"eval"`) : An optional prefix to be used as the metrics key prefix. For example the metrics "bleu" will be named "eval_bleu" if the prefix is `"eval"` (default)
max_length (`int`, *optional*) : The maximum target length to use when predicting with the generate method.

num_beams (`int`, *optional*) : Number of beams for beam search that will be used when predicting with the generate method. 1 means no beam search.

gen_kwargs : Additional `generate` specific kwargs.

**Returns:**

A dictionary containing the evaluation loss and the potential metrics computed from the predictions. The
dictionary also contains the epoch number which comes from the training state.
#### predict[[transformers.Seq2SeqTrainer.predict]]

[Source](https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/trainer_seq2seq.py#L195)

Run prediction and returns predictions and potential metrics.

Depending on the dataset and your use case, your test dataset may contain labels. In that case, this method
will also return metrics, like in `evaluate()`.

If your predictions or labels have different sequence lengths (for instance because you're doing dynamic
padding in a token classification task) the predictions will be padded (on the right) to allow for
concatenation into one array. The padding index is -100.

Returns: *NamedTuple* A namedtuple with the following keys:

- predictions (`np.ndarray`): The predictions on `test_dataset`.
- label_ids (`np.ndarray`, *optional*): The labels (if the dataset contained some).
- metrics (`dict[str, float]`, *optional*): The potential dictionary of metrics (if the dataset contained
  labels).

**Parameters:**
test_dataset (`Dataset`) : Dataset to run the predictions on. If it is a [Dataset](https://huggingface.co/docs/datasets/v4.8.4/en/package_reference/main_classes#datasets.Dataset), columns not accepted by the `model.forward()` method are automatically removed. Has to implement the method `__len__`

ignore_keys (`list[str]`, *optional*) : A list of keys in the output of your model (if it is a dictionary) that should be ignored when gathering predictions.

metric_key_prefix (`str`, *optional*, defaults to `"eval"`) : An optional prefix to be used as the metrics key prefix. For example the metrics "bleu" will be named "eval_bleu" if the prefix is `"eval"` (default)

max_length (`int`, *optional*) : The maximum target length to use when predicting with the generate method.

num_beams (`int`, *optional*) : Number of beams for beam search that will be used when predicting with the generate method. 1 means no beam search.

gen_kwargs : Additional `generate` specific kwargs.
#### transformers.TrainingArguments[[transformers.TrainingArguments]]

[Source](https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/training_args.py#L179)

Configuration class for controlling all aspects of model training with the Trainer.
TrainingArguments centralizes all hyperparameters, optimization settings, logging preferences, and infrastructure choices needed for training.

[HfArgumentParser](/docs/transformers/v5.6.2/en/internal/trainer_utils#transformers.HfArgumentParser) can turn this class into
[argparse](https://docs.python.org/3/library/argparse#module-argparse) arguments that can be specified on the
command line.

get_process_log_leveltransformers.TrainingArguments.get_process_log_levelhttps://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/training_args.py#L2018[]

Returns the log level to be used depending on whether this process is the main process of node 0, main process
of node non-0, or a non-main process.
For the main process the log level defaults to the logging level set (`logging.WARNING` if you didn't do
anything) unless overridden by `log_level` argument.

For the replica processes the log level defaults to `logging.WARNING` unless overridden by `log_level_replica`
argument.

The choice between the main and replica process settings is made according to the return value of `should_log`.

**Parameters:**

output_dir (`str`, *optional*, defaults to `"trainer_output"`) : The output directory where the model predictions and checkpoints will be written.
#### main_process_first[[transformers.TrainingArguments.main_process_first]]

[Source](https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/training_args.py#L2056)

A context manager for torch distributed environment where on needs to do something on the main process, while
blocking replicas, and when it's finished releasing the replicas.

One such use is for `datasets`'s `map` feature which to be efficient should be run once on the main process,
which upon completion saves a cached version of results and which then automatically gets loaded by the
replicas.

**Parameters:**
local (`bool`, *optional*, defaults to `True`) : if `True` first means process of rank 0 of each node if `False` first means process of rank 0 of node rank 0 In multi-node environment with a shared filesystem you most likely will want to use `local=False` so that only the main process of the first node will do the processing. If however, the filesystem is not shared, then the main process of each node will need to do the processing, which is the default behavior.

desc (`str`, *optional*, defaults to `"work"`) : a work description to be used in debug logs
#### set_dataloader[[transformers.TrainingArguments.set_dataloader]]

[Source](https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/training_args.py#L2641)

A method that regroups all arguments linked to the dataloaders creation.

Example:

```py
>>> from transformers import TrainingArguments

>>> args = TrainingArguments("working_dir")
>>> args = args.set_dataloader(train_batch_size=16, eval_batch_size=64)
>>> args.per_device_train_batch_size
16
```

**Parameters:**

drop_last (`bool`, *optional*, defaults to `False`) : Whether to drop the last incomplete batch (if the length of the dataset is not divisible by the batch size) or not.

num_workers (`int`, *optional*, defaults to 0) : Number of subprocesses to use for data loading (PyTorch only). 0 means that the data will be loaded in the main process.

pin_memory (`bool`, *optional*, defaults to `True`) : Whether you want to pin memory in data loaders or not. Will default to `True`.
persistent_workers (`bool`, *optional*, defaults to `False`) : If True, the data loader will not shut down the worker processes after a dataset has been consumed once. This allows to maintain the workers Dataset instances alive. Can potentially speed up training, but will increase RAM usage. Will default to `False`.

prefetch_factor (`int`, *optional*) : Number of batches loaded in advance by each worker. 2 means there will be a total of 2 * num_workers batches prefetched across all workers.

auto_find_batch_size (`bool`, *optional*, defaults to `False`) : Whether to find a batch size that will fit into memory automatically through exponential decay, avoiding CUDA Out-of-Memory errors. Requires accelerate to be installed (`pip install accelerate`)
ignore_data_skip (`bool`, *optional*, defaults to `False`) : When resuming training, whether or not to skip the epochs and batches to get the data loading at the same stage as in the previous training. If set to `True`, the training will begin faster (as that skipping step can take a long time) but will not yield the same results as the interrupted training would have.

sampler_seed (`int`, *optional*) : Random seed to be used with data samplers. If not set, random generators for data sampling will use the same seed as `self.seed`. This can be used to ensure reproducibility of data sampling, independent of the model seed.
#### set_evaluate[[transformers.TrainingArguments.set_evaluate]]

[Source](https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/training_args.py#L2253)

A method that regroups all arguments linked to evaluation.

Example:

```py
>>> from transformers import TrainingArguments

>>> args = TrainingArguments("working_dir")
>>> args = args.set_evaluate(strategy="steps", steps=100)
>>> args.eval_steps
100
```

**Parameters:**

strategy (`str` or [IntervalStrategy](/docs/transformers/v5.6.2/en/internal/trainer_utils#transformers.IntervalStrategy), *optional*, defaults to `"no"`) : The evaluation strategy to adopt during training. Possible values are:  - `"no"`: No evaluation is done during training. - `"steps"`: Evaluation is done (and logged) every `steps`. - `"epoch"`: Evaluation is done at the end of each epoch.  Setting a `strategy` different from `"no"` will set `self.do_eval` to `True`.
steps (`int`, *optional*, defaults to 500) : Number of update steps between two evaluations if `strategy="steps"`.

batch_size (`int` *optional*, defaults to 8) : The batch size per device (GPU/TPU core/CPU...) used for evaluation.

accumulation_steps (`int`, *optional*) : Number of predictions steps to accumulate the output tensors for, before moving the results to the CPU. If left unset, the whole predictions are accumulated on GPU/TPU before being moved to the CPU (faster but requires more memory).

delay (`float`, *optional*) : Number of epochs or steps to wait for before the first evaluation can be performed, depending on the eval_strategy.

loss_only (`bool`, *optional*, defaults to `False`) : Ignores all outputs except the loss.
#### set_logging[[transformers.TrainingArguments.set_logging]]

[Source](https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/training_args.py#L2395)

A method that regroups all arguments linked to logging.

Example:

```py
>>> from transformers import TrainingArguments

>>> args = TrainingArguments("working_dir")
>>> args = args.set_logging(strategy="steps", steps=100)
>>> args.logging_steps
100
```

**Parameters:**

strategy (`str` or [IntervalStrategy](/docs/transformers/v5.6.2/en/internal/trainer_utils#transformers.IntervalStrategy), *optional*, defaults to `"steps"`) : The logging strategy to adopt during training. Possible values are:  - `"no"`: No logging is done during training. - `"epoch"`: Logging is done at the end of each epoch. - `"steps"`: Logging is done every `logging_steps`.

steps (`int`, *optional*, defaults to 500) : Number of update steps between two logs if `strategy="steps"`.
level (`str`, *optional*, defaults to `"passive"`) : Logger log level to use on the main process. Possible choices are the log levels as strings: `"debug"`, `"info"`, `"warning"`, `"error"` and `"critical"`, plus a `"passive"` level which doesn't set anything and lets the application set the level.

report_to (`str` or `list[str]`, *optional*, defaults to `"none"`) : The list of integrations to report the results and logs to. Supported platforms are `"azure_ml"`, `"clearml"`, `"codecarbon"`, `"comet_ml"`, `"dagshub"`, `"dvclive"`, `"flyte"`, `"mlflow"`, `"swanlab"`, `"tensorboard"`, `"trackio"` and `"wandb"`. Use `"all"` to report to all integrations installed, `"none"` for no integrations.

first_step (`bool`, *optional*, defaults to `False`) : Whether to log and evaluate the first `global_step` or not.
nan_inf_filter (`bool`, *optional*, defaults to `True`) : Whether to filter `nan` and `inf` losses for logging. If set to `True` the loss of every step that is `nan` or `inf` is filtered and the average loss of the current logging window is taken instead.    `nan_inf_filter` only influences the logging of loss values, it does not change the behavior the gradient is computed or applied to the model.   

on_each_node (`bool`, *optional*, defaults to `True`) : In multinode distributed training, whether to log using `log_level` once per node, or only on the main node.

replica_level (`str`, *optional*, defaults to `"passive"`) : Logger log level to use on replicas. Same choices as `log_level`
#### set_lr_scheduler[[transformers.TrainingArguments.set_lr_scheduler]]

[Source](https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/training_args.py#L2595)

A method that regroups all arguments linked to the learning rate scheduler and its hyperparameters.

Example:

```py
>>> from transformers import TrainingArguments

>>> args = TrainingArguments("working_dir")
>>> args = args.set_lr_scheduler(name="cosine", warmup_steps=0.05)
>>> args.warmup_steps
0.05
```

**Parameters:**

name (`str` or [SchedulerType](/docs/transformers/v5.6.2/en/main_classes/optimizer_schedules#transformers.SchedulerType), *optional*, defaults to `"linear"`) : The scheduler type to use. See the documentation of [SchedulerType](/docs/transformers/v5.6.2/en/main_classes/optimizer_schedules#transformers.SchedulerType) for all possible values.
num_epochs(`float`, *optional*, defaults to 3.0) : Total number of training epochs to perform (if not an integer, will perform the decimal part percents of the last epoch before stopping training).

max_steps (`int`, *optional*, defaults to -1) : If set to a positive number, the total number of training steps to perform. Overrides `num_train_epochs`. For a finite dataset, training is reiterated through the dataset (if all data is exhausted) until `max_steps` is reached.

warmup_steps (`float`, *optional*, defaults to 0) : Number of steps used for a linear warmup from 0 to `learning_rate`.  Should be an integer or a float in range `[0,1)`. If smaller than 1, will be interpreted as ratio of steps used for a linear warmup from 0 to `learning_rate`.
#### set_optimizer[[transformers.TrainingArguments.set_optimizer]]

[Source](https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/training_args.py#L2544)

A method that regroups all arguments linked to the optimizer and its hyperparameters.

Example:

```py
>>> from transformers import TrainingArguments

>>> args = TrainingArguments("working_dir")
>>> args = args.set_optimizer(name="adamw_torch", beta1=0.8)
>>> args.optim
'adamw_torch'
```

**Parameters:**

name (`str` or `training_args.OptimizerNames`, *optional*, defaults to `"adamw_torch"`) : The optimizer to use: `"adamw_torch"`, `"adamw_torch_fused"`, `"adamw_apex_fused"`, `"adamw_anyprecision"` or `"adafactor"`.

learning_rate (`float`, *optional*, defaults to 5e-5) : The initial learning rate.

weight_decay (`float`, *optional*, defaults to 0) : The weight decay to apply (if not zero) to all layers except all bias and LayerNorm weights.
beta1 (`float`, *optional*, defaults to 0.9) : The beta1 hyperparameter for the adam optimizer or its variants.

beta2 (`float`, *optional*, defaults to 0.999) : The beta2 hyperparameter for the adam optimizer or its variants.

epsilon (`float`, *optional*, defaults to 1e-8) : The epsilon hyperparameter for the adam optimizer or its variants.

args (`str`, *optional*) : Optional arguments that are supplied to AnyPrecisionAdamW (only useful when `optim="adamw_anyprecision"`).
#### set_push_to_hub[[transformers.TrainingArguments.set_push_to_hub]]

[Source](https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/training_args.py#L2470)

A method that regroups all arguments linked to synchronizing checkpoints with the Hub.

Calling this method will set `self.push_to_hub` to `True`, which means the `output_dir` will begin a git
directory synced with the repo (determined by `model_id`) and the content will be pushed each time a save is
triggered (depending on your `self.save_strategy`). Calling [save_model()](/docs/transformers/v5.6.2/en/main_classes/trainer#transformers.Trainer.save_model) will also trigger a push.

Example:

```py
>>> from transformers import TrainingArguments

>>> args = TrainingArguments("working_dir")
>>> args = args.set_push_to_hub("me/awesome-model")
>>> args.hub_model_id
'me/awesome-model'
```

**Parameters:**
strategy (`str` or `HubStrategy`, *optional*, defaults to `"every_save"`) : Defines the scope of what is pushed to the Hub and when. Possible values are:  - `"end"`: push the model, its configuration, the processing_class e.g. tokenizer (if passed along to the [Trainer](/docs/transformers/v5.6.2/en/main_classes/trainer#transformers.Trainer)) and a draft of a model card when the [save_model()](/docs/transformers/v5.6.2/en/main_classes/trainer#transformers.Trainer.save_model) method is called. - `"every_save"`: push the model, its configuration, the processing_class e.g. tokenizer (if passed along to the [Trainer](/docs/transformers/v5.6.2/en/main_classes/trainer#transformers.Trainer)) and a draft of a model card each time there is a model save. The pushes are asynchronous to not block training, and in case the save are very frequent, a new push is only attempted if the previous one is finished. A last push is made with the final model at the end of training. - `"checkpoint"`: like `"every_save"` but the latest checkpoint is also pushed in a subfolder named last-checkpoint, allowing you to resume training easily with `trainer.train(resume_from_checkpoint="last-checkpoint")`. - `"all_checkpoints"`: like `"checkpoint"` but all checkpoints are pushed like they appear in the output folder (so you will get one checkpoint folder per folder in your final repository)
token (`str`, *optional*) : The token to use to push the model to the Hub. Will default to the token in the cache folder obtained with `hf auth login`.

private_repo (`bool`, *optional*, defaults to `False`) : Whether to make the repo private. If `None` (default), the repo will be public unless the organization's default is private. This value is ignored if the repo already exists.

always_push (`bool`, *optional*, defaults to `False`) : Unless this is `True`, the `Trainer` will skip pushing a checkpoint when the previous push is not finished.

revision (`str`, *optional*) : The revision to use when pushing to the Hub. Can be a branch name, a tag, or a commit hash.
#### set_save[[transformers.TrainingArguments.set_save]]

[Source](https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/training_args.py#L2346)

A method that regroups all arguments linked to checkpoint saving.

Example:

```py
>>> from transformers import TrainingArguments

>>> args = TrainingArguments("working_dir")
>>> args = args.set_save(strategy="steps", steps=100)
>>> args.save_steps
100
```

**Parameters:**

strategy (`str` or [IntervalStrategy](/docs/transformers/v5.6.2/en/internal/trainer_utils#transformers.IntervalStrategy), *optional*, defaults to `"steps"`) : The checkpoint save strategy to adopt during training. Possible values are:  - `"no"`: No save is done during training. - `"epoch"`: Save is done at the end of each epoch. - `"steps"`: Save is done every `save_steps`.

steps (`int`, *optional*, defaults to 500) : Number of updates steps before two checkpoint saves if `strategy="steps"`.
total_limit (`int`, *optional*) : If a value is passed, will limit the total amount of checkpoints. Deletes the older checkpoints in `output_dir`.

on_each_node (`bool`, *optional*, defaults to `False`) : When doing multi-node distributed training, whether to save models and checkpoints on each node, or only on the main one.  This should not be activated when the different nodes use the same storage as the files will be saved with the same names for each node.
#### set_testing[[transformers.TrainingArguments.set_testing]]

[Source](https://github.com/huggingface/transformers/blob/v5.6.2/src/transformers/training_args.py#L2310)

A method that regroups all basic arguments linked to testing on a held-out dataset.

Calling this method will automatically set `self.do_predict` to `True`.

Example:

```py
>>> from transformers import TrainingArguments

>>> args = TrainingArguments("working_dir")
>>> args = args.set_testing(batch_size=32)
>>> args.per_device_eval_batch_size
32
```

**Parameters:**

batch_size (`int` *optional*, defaults to 8) : The batch size per device (GPU/TPU core/CPU...) used for testing.

loss_only (`bool`, *optional*, defaults to `False`) : Ignores all outputs except the loss.