IPUTrainer

The IPUTrainer class provides a similar API than the Transformers Trainer to perform training, evaluation and prediction on Graphcore’s IPUs. It is the class used in all the example scripts.

Compared to the regular Transformers Trainer, to instantiate a IPUTrainer you need to create:

An instance of IPUTrainingArguments, which allows you to customize the behaviour of the trainer
An instance of IPUConfig, which specifies IPU specific parameters

IPUTrainer

class optimum.graphcore.IPUTrainer

< source >

( model: typing.Union[transformers.modeling_utils.PreTrainedModel, torch.nn.modules.module.Module] = None ipu_config: IPUConfig = None args: IPUTrainingArguments = None data_collator: typing.Optional[DataCollator] = None eval_data_collator: typing.Optional[DataCollator] = None train_dataset: typing.Optional[torch.utils.data.dataset.Dataset] = None eval_dataset: typing.Optional[torch.utils.data.dataset.Dataset] = None tokenizer: typing.Optional[transformers.tokenization_utils_base.PreTrainedTokenizerBase] = None model_init: typing.Callable[[], transformers.modeling_utils.PreTrainedModel] = None compute_metrics: typing.Union[typing.Callable[[transformers.trainer_utils.EvalPrediction], typing.Dict], NoneType] = None callbacks: typing.Optional[typing.List[transformers.trainer_callback.TrainerCallback]] = None optimizers: typing.Tuple[torch.optim.optimizer.Optimizer, torch.optim.lr_scheduler.LambdaLR] = (None, None) preprocess_logits_for_metrics: typing.Union[typing.Callable[[torch.Tensor, torch.Tensor], torch.Tensor], NoneType] = None force_to_pipelined: bool = False )

Parameters

model (transformers.PreTrainedModel or torch.nn.Module, optional) — The model to train, evaluate or use for predictions. If not provided, a model_init must be passed.

IPUTrainer is optimized to work with the transformers.PreTrainedModel provided by the 🤗 Transformers library. You can still use your own models defined as torch.nn.Module as long as they work the same way as the 🤗 Transformers models.
args (IPUTrainingArguments, optional) — The arguments to tweak for training. Will default to a basic instance of IPUTrainingArguments with the output_dir set to a directory named tmp_trainer in the current directory if not provided.
data_collator (transformers.data.data_collator.DataCollator, optional) — The function to use to form a batch from a list of elements of train_dataset or eval_dataset. Will default to transformers.data.default_data_collator if no tokenizer is provided, an instance of DataCollatorWithPadding otherwise.
train_dataset (torch.utils.data.Dataset or torch.utils.data.IterableDataset, optional) — The dataset to use for training. If it is a Dataset, columns not accepted by the model.forward() method are automatically removed.

Note that if it’s a torch.utils.data.IterableDataset with some randomization and you are training in a distributed fashion, your iterable dataset should either use a internal attribute generator that is a torch.Generator for the randomization that must be identical on all processes (and the Trainer will manually set the seed of this generator at each epoch) or have a set_epoch() method that internally sets the seed of the RNGs used.
eval_dataset (Union[torch.utils.data.Dataset, Dict[str, torch.utils.data.Dataset]), optional) — The dataset to use for evaluation. If it is a Dataset, columns not accepted by the model.forward() method are automatically removed. If it is a dictionary, it will evaluate on each dataset prepending the dictionary key to the metric name.
tokenizer (transformers.PreTrainedTokenizerBase, optional) — The tokenizer used to preprocess the data. If provided, will be used to automatically pad the inputs the maximum length when batching inputs, and it will be saved along the model to make it easier to rerun an interrupted training or reuse the fine-tuned model.
model_init (Callable[[], transformers.PreTrainedModel], optional) — A function that instantiates the model to be used. If provided, each call to IPUTrainer.train() will start from a new instance of the model as given by this function.

The function may have zero argument, or a single one containing the optuna/Ray Tune/SigOpt trial object, to be able to choose different architectures according to hyper parameters (such as layer count, sizes of inner layers, dropout probabilities etc). Note: this feature is not supported for now.
compute_metrics (Callable[[~transformers.trainer_utils.EvalPrediction], Dict], optional) — The function that will be used to compute metrics at evaluation. Must take a EvalPrediction and return a dictionary string to metric values.
callbacks (List of transformers.trainer_callback.TrainerCallback, optional) — A list of callbacks to customize the training loop. Will add those to the list of default callbacks detailed in here.

If you want to remove one of the default callbacks used, use the Trainer.remove_callback method.
optimizers (Tuple[torch.optim.Optimizer, torch.optim.lr_scheduler.LambdaLR], optional) — A tuple containing the optimizer and the scheduler to use. Will default to an instance of poptorch.AdamW on your model and a scheduler given by get_linear_schedule_with_warmup controlled by args.
preprocess_logits_for_metrics (Callable[[torch.Tensor, torch.Tensor], torch.Tensor], optional) — A function that preprocess the logits right before caching them at each evaluation step. Must take two tensors, the logits and the labels, and return the logits once processed as desired. The modifications made by this function will be reflected in the predictions received by compute_metrics.

Note that the labels (second parameter) will be None if the dataset does not have them.

IPUTrainer is a simple but feature-complete training and eval loop on Graphcore IPUs for PyTorch, optimized for 🤗 Transformers.

add_callback

< source >

( callback )

Parameters

callback (type or ~transformer.TrainerCallback) — A ~transformer.TrainerCallback class or an instance of a ~transformer.TrainerCallback. In the first case, will instantiate a member of that class.

Adds a callback to the current list of ~transformer.TrainerCallback.

compile_model

< source >

( model: PoplarExecutor sample_batch: typing.Union[typing.Dict[str, torch.Tensor], typing.Tuple[torch.Tensor]] log: bool = False )

Parameters

model (poptorch.PoplarExecutor) — The model to compile (already wrapped).
sample_batch (Dict[str, torch.Tensor] or Tuple[torch.Tensor]) — The inputs to use the compilation, this will set the input shapes that the compiled model can accept.
log (bool, optional, defaults to False) — Whether to log that compilation is happening or not.

Compiles the model with PopTorch.

compute_loss

< source >

( model inputs return_outputs = False )

How the loss is computed by IPUTrainer. By default, all models return the loss in the first element.

Subclass and override for custom behavior.

create_model_card

< source >

( language: typing.Optional[str] = None license: typing.Optional[str] = None tags: typing.Union[str, typing.List[str], NoneType] = None model_name: typing.Optional[str] = None finetuned_from: typing.Optional[str] = None tasks: typing.Union[str, typing.List[str], NoneType] = None dataset_tags: typing.Union[str, typing.List[str], NoneType] = None dataset: typing.Union[str, typing.List[str], NoneType] = None dataset_args: typing.Union[str, typing.List[str], NoneType] = None )

Parameters

language (str, optional) — The language of the model (if applicable)
license (str, optional) — The license of the model. Will default to the license of the pretrained model used, if the original model given to the Trainer comes from a repo on the Hub.
tags (str or List[str], optional) — Some tags to be included in the metadata of the model card.
model_name (str, optional) — The name of the model.
finetuned_from (str, optional) — The name of the model used to fine-tune this one (if applicable). Will default to the name of the repo of the original model given to the Trainer (if it comes from the Hub).
tasks (str or List[str], optional) — One or several task identifiers, to be included in the metadata of the model card.
dataset_tags (str or List[str], optional) — One or several dataset tags, to be included in the metadata of the model card.
dataset (str or List[str], optional) — One or several dataset identifiers, to be included in the metadata of the model card.
dataset_args (str or List[str], optional) — One or several dataset arguments, to be included in the metadata of the model card.

Creates a draft of a model card using the information available to the Trainer.

create_optimizer

< source >

( )

Setup the optimizer.

We provide a reasonable default that works well. If you want to use something else, you can pass a tuple in the Trainer’s init through optimizers, or subclass and override this method in a subclass.

create_optimizer_and_scheduler

< source >

( num_training_steps: int )

Setup the optimizer and the learning rate scheduler.

create_scheduler

< source >

( num_training_steps: int optimizer: Optimizer = None )

Parameters

num_training_steps (int) — The number of training steps to do.

Setup the scheduler. The optimizer of the trainer must have been set up either before this method is called or passed as an argument.

evaluate

< source >

( eval_dataset: typing.Optional[torch.utils.data.dataset.Dataset] = None ignore_keys: typing.Optional[typing.List[str]] = None metric_key_prefix: str = 'eval' )

Parameters

eval_dataset (Dataset, optional) — Pass a dataset if you wish to override self.eval_dataset. If it is a Dataset, columns not accepted by the model.forward() method are automatically removed. It must implement the __len__ method.
ignore_keys (Lst[str], optional) — A list of keys in the output of your model (if it is a dictionary) that should be ignored when gathering predictions.
metric_key_prefix (str, optional, defaults to "eval") — An optional prefix to be used as the metrics key prefix. For example the metrics “bleu” will be named “eval_bleu” if the prefix is “eval” (default)

Run evaluation and returns metrics.

The calling script will be responsible for providing a method to compute metrics, as they are task-dependent (pass it to the init compute_metrics argument).

You can also subclass and override this method to inject custom behavior.

evaluation_loop

< source >

( dataloader: DataLoader description: str prediction_loss_only: typing.Optional[bool] = None ignore_keys: typing.Optional[typing.List[str]] = None metric_key_prefix: str = 'eval' )

Prediction/evaluation loop, shared by IPUTrainer.evaluate() and IPUTrainer.predict().

Works both with or without labels.

floating_point_ops

< source >

( inputs: typing.Dict[str, typing.Union[torch.Tensor, typing.Any]] ) → int

Parameters

inputs (Dict[str, Union[torch.Tensor, Any]]) — The inputs and targets of the model.

Returns

int

The number of floating-point operations.

For models that inherit from transformers.PreTrainedModel, uses that method to compute the number of floating point operations for every backward + forward pass. If using another model, either implement such a method in the model or subclass and override this method.

get_eval_dataloader

< source >

( eval_dataset: typing.Optional[torch.utils.data.dataset.Dataset] = None )

Parameters

eval_dataset (torch.utils.data.Dataset, optional) — If provided, will override self.eval_dataset. If it is a Dataset, columns not accepted by the model.forward() method are automatically removed. It must implement __len__.

Returns the evaluation poptorch.DataLoader.

Subclass and override this method if you want to inject some custom behavior.

get_test_dataloader

< source >

( test_dataset: Dataset )

Parameters

test_dataset (torch.utils.data.Dataset, optional) — The test dataset to use. If it is a Dataset, columns not accepted by the model.forward() method are automatically removed. It must implement __len__.

Returns the test poptorch.DataLoader.

Subclass and override this method if you want to inject some custom behavior.

get_train_dataloader

< source >

( )

Returns the training poptorch.DataLoader.

Will use no sampler if train_dataset does not implement __len__, a random sampler (adapted to distributed training if necessary) otherwise.

Subclass and override this method if you want to inject some custom behavior.

init_git_repo

< source >

( at_init: bool = False )

Parameters

at_init (bool, optional, defaults to False) — Whether this function is called before any training or not. If self.args.overwrite_output_dir is True and at_init is True, the path to the repo (which is self.args.output_dir) might be wiped out.

Initializes a git repo in self.args.hub_model_id.

log

< source >

( logs: typing.Dict[str, float] )

Parameters

logs (Dict[str, float]) — The values to log.

Log logs on the various objects watching training.

Subclass and override this method to inject custom behavior.

log_metrics

< source >

( split metrics )

Parameters

split (str) — Mode/split name: one of train, eval, test
metrics (Dict[str, float]) — The metrics returned from train/evaluate/predictmetrics: metrics dict

Log metrics in a specially formatted way

Under distributed environment this is done only for a process with rank 0.

Notes on memory reports:

In order to get memory usage report you need to install psutil. You can do that with pip install psutil.

Now when this method is run, you will see a report that will include: :

init_mem_cpu_alloc_delta   =     1301MB
init_mem_cpu_peaked_delta  =      154MB
init_mem_gpu_alloc_delta   =      230MB
init_mem_gpu_peaked_delta  =        0MB
train_mem_cpu_alloc_delta  =     1345MB
train_mem_cpu_peaked_delta =        0MB
train_mem_gpu_alloc_delta  =      693MB
train_mem_gpu_peaked_delta =        7MB

Understanding the reports:

the first segment, e.g., train__, tells you which stage the metrics are for. Reports starting with init_ will be added to the first stage that gets run. So that if only evaluation is run, the memory usage for the __init__ will be reported along with the eval_ metrics.
the third segment, is either cpu or gpu, tells you whether it’s the general RAM or the gpu0 memory metric.
*_alloc_delta - is the difference in the used/allocated memory counter between the end and the start of the stage - it can be negative if a function released more memory than it allocated.
*_peaked_delta - is any extra memory that was consumed and then freed - relative to the current allocated memory counter - it is never negative. When you look at the metrics of any stage you add up alloc_delta + peaked_delta and you know how much memory was needed to complete that stage.

The reporting happens only for process of rank 0 and gpu 0 (if there is a gpu). Typically this is enough since the main process does the bulk of work, but it could be not quite so if model parallel is used and then other GPUs may use a different amount of gpu memory. This is also not the same under DataParallel where gpu0 may require much more memory than the rest since it stores the gradient and optimizer states for all participating GPUS. Perhaps in the future these reports will evolve to measure those too.

The CPU RAM metric measures RSS (Resident Set Size) includes both the memory which is unique to the process and the memory shared with other processes. It is important to note that it does not include swapped out memory, so the reports could be imprecise.

The CPU peak memory is measured using a sampling thread. Due to python’s GIL it may miss some of the peak memory if that thread didn’t get a chance to run when the highest memory was used. Therefore this report can be less than reality. Using tracemalloc would have reported the exact peak memory, but it doesn’t report memory allocations outside of python. So if some C++ CUDA extension allocated its own memory it won’t be reported. And therefore it was dropped in favor of the memory sampling approach, which reads the current process memory usage.

The GPU allocated and peak memory reporting is done with torch.cuda.memory_allocated() and torch.cuda.max_memory_allocated(). This metric reports only “deltas” for pytorch-specific allocations, as torch.cuda memory management system doesn’t track any memory allocated outside of pytorch. For example, the very first cuda call typically loads CUDA kernels, which may take from 0.5 to 2GB of GPU memory.

Note that this tracker doesn’t account for memory allocations outside of Trainer’s __init__, train, evaluate and predict calls.

Because evaluation calls may happen during train, we can’t handle nested invocations because torch.cuda.max_memory_allocated is a single counter, so if it gets reset by a nested eval call, train’s tracker will report incorrect info. If this pytorch issue gets resolved it will be possible to change this class to be re-entrant. Until then we will only track the outer level of train, evaluate and predict methods. Which means that if eval is called during train, it’s the latter that will account for its memory usage and that of the former.

This also means that if any other tool that is used along the Trainer calls torch.cuda.reset_peak_memory_stats, the gpu peak memory stats could be invalid. And the Trainer will disrupt the normal behavior of any such tools that rely on calling torch.cuda.reset_peak_memory_stats themselves.

For best performance you may want to consider turning the memory profiling off for production runs.

metrics_format

< source >

( metrics: typing.Dict[str, float] ) → metrics (Dict[str, float])

Parameters

metrics (Dict[str, float]) — The metrics returned from train/evaluate/predict

Returns

metrics (Dict[str, float])

The reformatted metrics

Reformat Trainer metrics values to a human-readable format

num_examples

< source >

( dataloader: DataLoader )

Helper to get number of samples in a poptorch.DataLoader by accessing its dataset. When dataloader.dataset does not exist or has no length, estimates as best it can

pop_callback

< source >

( callback ) → ~transformer.TrainerCallback

Parameters

callback (type or ~transformer.TrainerCallback) — A ~transformer.TrainerCallback class or an instance of a ~transformer.TrainerCallback. In the first case, will pop the first member of that class found in the list of callbacks.

Returns

~transformer.TrainerCallback

The callback removed, if found.

Removes a callback from the current list of ~transformer.TrainerCallback and returns it.

If the callback is not found, returns None (and no error is raised).

predict

< source >

( test_dataset: Dataset ignore_keys: typing.Optional[typing.List[str]] = None metric_key_prefix: str = 'test' )

Parameters

test_dataset (Dataset) — Dataset to run the predictions on. If it is an datasets.Dataset, columns not accepted by the model.forward() method are automatically removed. Has to implement the method __len__
ignore_keys (Lst[str], optional) — A list of keys in the output of your model (if it is a dictionary) that should be ignored when gathering predictions.
metric_key_prefix (str, optional, defaults to "test") — An optional prefix to be used as the metrics key prefix. For example the metrics “bleu” will be named “test_bleu” if the prefix is “test” (default)

Run prediction and returns predictions and potential metrics.

Depending on the dataset and your use case, your test dataset may contain labels. In that case, this method will also return metrics, like in evaluate().

If your predictions or labels have different sequence length (for instance because you’re doing dynamic padding in a token classification task) the predictions will be padded (on the right) to allow for concatenation into one array. The padding index is -100.

Returns: NamedTuple A namedtuple with the following keys:

predictions (np.ndarray): The predictions on test_dataset.
label_ids (np.ndarray, optional): The labels (if the dataset contained some).
metrics (Dict[str, float], optional): The potential dictionary of metrics (if the dataset contained labels).

prediction_step

< source >

( model: PoplarExecutor inputs: typing.Dict[str, typing.Union[torch.Tensor, typing.Any]] prediction_loss_only: bool ignore_keys: typing.Optional[typing.List[str]] = None is_last_batch: bool = False ) → Tuple[Optional[torch.Tensor], Optional[torch.Tensor], Optional[torch.Tensor]]

Parameters

model (poptorch.PoplarExecutor) — The model to evaluate.
inputs (Dict[str, Union[torch.Tensor, Any]]) — The inputs and targets of the model.

The dictionary will be unpacked before being fed to the model. Most models expect the targets under the argument labels. Check your model’s documentation for all accepted arguments.
prediction_loss_only (bool) — Whether or not to return the loss only.
ignore_keys (Lst[str], optional) — A list of keys in the output of your model (if it is a dictionary) that should be ignored when gathering predictions.

Returns

Tuple[Optional[torch.Tensor], Optional[torch.Tensor], Optional[torch.Tensor]]

A tuple with the loss, logits and labels (each being optional).

Perform an evaluation step on model using inputs.

Subclass and override to inject custom behavior.

push_to_hub

< source >

( commit_message: typing.Optional[str] = 'End of training' blocking: bool = True **kwargs )

Parameters

commit_message (str, optional, defaults to "End of training") — Message to commit while pushing.
blocking (bool, optional, defaults to True) — Whether the function should return only when the git push has finished. kwargs — Additional keyword arguments passed along to ~Trainer.create_model_card.

Upload self.model and self.tokenizer to the 🤗 model hub on the repo self.args.hub_model_id.

pytorch_optimizer_to_poptorch

< source >

( optimizer: Optimizer model: typing.Union[transformers.modeling_utils.PreTrainedModel, torch.nn.modules.module.Module] pipelined_model: typing.Union[transformers.modeling_utils.PreTrainedModel, torch.nn.modules.module.Module] ) → poptorch.optim.Optimizer

Parameters

optimizer (torch.optim.Optimizer) — The PyTorch optimizer to convert.
model ([transformers.PreTrainedModel] or torch.nn.Module) — The original model the optimizer has parameter references to.
pipelined_model ([transformers.PreTrainedModel] or torch.nn.Module`) — The pipelined version of the model, its parameters will be used by the poptorch optimizer.

Returns

poptorch.optim.Optimizer

The converted poptorch optimizer.

Converts a PyTorch optimizer to a PopTorch optimizer.

remove_callback

< source >

( callback )

Parameters

callback (type or ~transformer.TrainerCallback) — A ~transformer.TrainerCallback class or an instance of a ~transformer.TrainerCallback. In the first case, will remove the first member of that class found in the list of callbacks.

Removes a callback from the current list of ~transformer.TrainerCallback.

save_metrics

< source >

( split metrics combined = True )

Parameters

split (str) — Mode/split name: one of train, eval, test, all
metrics (Dict[str, float]) — The metrics returned from train/evaluate/predict
combined (bool, optional, defaults to True) — Creates combined metrics by updating all_results.json with metrics of this call

Save metrics into a json file for that split, e.g. train_results.json.

Under distributed environment this is done only for a process with rank 0.

To understand the metrics please read the docstring of ~Trainer.log_metrics. The only difference is that raw unformatted numbers are saved in the current method.

save_model

< source >

( output_dir: typing.Optional[str] = None _internal_call: bool = False )

Will save the model, so you can reload it using from_pretrained().

Will only save from the main process.

save_state

< source >

( )

Saves the Trainer state, since Trainer.save_model saves only the tokenizer with the model

Under distributed environment this is done only for a process with rank 0.

train

< source >

( resume_from_checkpoint: typing.Union[str, bool, NoneType] = None trial: typing.Union[ForwardRef('optuna.Trial'), typing.Dict[str, typing.Any]] = None ignore_keys_for_eval: typing.Optional[typing.List[str]] = None **kwargs )

Parameters

resume_from_checkpoint (str or bool, optional) — If a str, local path to a saved checkpoint as saved by a previous instance of IPUTrainer. If a bool and equals True, load the last checkpoint in args.output_dir as saved by a previous instance of Trainer. If present, training will resume from the model/optimizer/scheduler states loaded here.
trial (optuna.Trial or Dict[str, Any], optional) — The trial run or the hyperparameter dictionary for hyperparameter search. Note: Feature not supported for now.
ignore_keys_for_eval (List[str], optional) — A list of keys in the output of your model (if it is a dictionary) that should be ignored when gathering predictions for evaluation during the training. kwargs — Additional keyword arguments used to hide deprecated arguments

Main training entry point.

training_step

< source >

( model: PoplarExecutor inputs: typing.Dict[str, typing.Union[torch.Tensor, typing.Any]] ) → torch.Tensor

Parameters

model (poptorch.PoplarExecutor) — The model to train.
inputs (Dict[str, Union[torch.Tensor, Any]]) — The inputs and targets of the model.

The dictionary will be unpacked before being fed to the model. Most models expect the targets under the argument labels. Check your model’s documentation for all accepted arguments.

Returns

torch.Tensor

The tensor with training loss on this batch.

Perform a training step on a batch of inputs.

Subclass and override to inject custom behavior.

wrap_model

< source >

( model: typing.Union[transformers.modeling_utils.PreTrainedModel, poptorch._poplar_executor.PoplarExecutor] training = True ) → poptorch.PoplarExecutor

Parameters

model (transformers.PreTrainedModel or poptorch.PoplarExecutor) — The model to wrap.
training (bool, optional, defaults to True) — Whether to wrap the model for training or not.

Returns

poptorch.PoplarExecutor

The wrapped model.

Wraps a model for PopTorch, either for training or for inference.

IPUSeq2SeqTrainer

The IPUSeq2SeqTrainer is used to train seq2seq models. It’s behaviour is exactly the same as IPUTrainer except that it expects IPUSeq2SeqTrainingArguments instead of IPUTrainingArguments.

class optimum.graphcore.IPUSeq2SeqTrainer

< source >

evaluate

< source >

( eval_dataset: typing.Optional[torch.utils.data.dataset.Dataset] = None ignore_keys: typing.Optional[typing.List[str]] = None metric_key_prefix: str = 'eval' max_length: typing.Optional[int] = None num_beams: typing.Optional[int] = None )

Parameters

eval_dataset (Dataset, optional) — Pass a dataset if you wish to override self.eval_dataset. If it is an datasets.Dataset, columns not accepted by the model.forward() method are automatically removed. It must implement the __len__ method.
ignore_keys (List[str], optional) — A list of keys in the output of your model (if it is a dictionary) that should be ignored when gathering predictions.
metric_key_prefix (str, optional, defaults to "eval") — An optional prefix to be used as the metrics key prefix. For example the metrics “bleu” will be named “eval_bleu” if the prefix is "eval" (default)
max_length (int, optional) — The maximum target length to use when predicting with the generate method.
num_beams (int, optional) — Number of beams for beam search that will be used when predicting with the generate method. 1 means no beam search.

Run evaluation and returns metrics.

The calling script will be responsible for providing a method to compute metrics, as they are task-dependent (pass it to the init compute_metrics argument).

You can also subclass and override this method to inject custom behavior.

predict

< source >

( test_dataset: Dataset ignore_keys: typing.Optional[typing.List[str]] = None metric_key_prefix: str = 'test' max_length: typing.Optional[int] = None num_beams: typing.Optional[int] = None )

Parameters

test_dataset (Dataset) — Dataset to run the predictions on. If it is an datasets.Dataset, columns not accepted by the model.forward() method are automatically removed. Has to implement the method __len__
ignore_keys (List[str], optional) — A list of keys in the output of your model (if it is a dictionary) that should be ignored when gathering predictions.
metric_key_prefix (str, optional, defaults to "eval") — An optional prefix to be used as the metrics key prefix. For example the metrics “bleu” will be named “eval_bleu” if the prefix is "eval" (default)
max_length (int, optional) — The maximum target length to use when predicting with the generate method.
num_beams (int, optional) — Number of beams for beam search that will be used when predicting with the generate method. 1 means no beam search.

Run prediction and returns predictions and potential metrics.

Depending on the dataset and your use case, your test dataset may contain labels. In that case, this method will also return metrics, like in evaluate().

If your predictions or labels have different sequence lengths (for instance because you’re doing dynamic padding in a token classification task) the predictions will be padded (on the right) to allow for concatenation into one array. The padding index is -100.

Returns: NamedTuple A namedtuple with the following keys:

predictions (np.ndarray): The predictions on test_dataset.
label_ids (np.ndarray, optional): The labels (if the dataset contained some).
metrics (Dict[str, float], optional): The potential dictionary of metrics (if the dataset contained labels).

Optimum

IPUTrainer

IPUTrainer

class optimum.graphcore.IPUTrainer

add_callback

compile_model

compute_loss

create_model_card

create_optimizer

create_optimizer_and_scheduler

create_scheduler

evaluate

evaluation_loop

floating_point_ops

get_eval_dataloader

get_test_dataloader

get_train_dataloader

init_git_repo

log

log_metrics

metrics_format

num_examples

pop_callback

predict

prediction_step

push_to_hub

pytorch_optimizer_to_poptorch

remove_callback

save_metrics

save_model

save_state

train

training_step

wrap_model

IPUSeq2SeqTrainer

class optimum.graphcore.IPUSeq2SeqTrainer

evaluate

predict