Parameters

• seed (int, optional) — Sets the seed for the random number generator on the IPU.
• auto_loss_scaling (bool, optional, defaults to False) — If True, enables automatic loss scaling on the IPU. When using float16/half-precision values for activations, gradients, and weights, the loss value needs to be scaled by a constant factor to avoid underflows or overflows. This adjustment is known as loss scaling. This setting automatically sets a global loss scaling factor during training. Note: This is an experimental feature and may not behave as expected.
• executable_cache_dir (str, optional, defaults to "") — Enables caching the compile executables to a directory.

Parameters for controlling the batch size

• replication_factor (int, optional, defaults to 1) — The number of replicas for data-parallelism during training. It depends on the size of the pipeline as well as the number of IPUs available. For example: on a Pod16, with a 4-IPU pipeline, the replication_factor must be between 1 and 4.
• inference_replication_factor (int, optional, defaults to 1) — The number of replicas for data-parallelism during inference. It depends on the size of the pipeline as well as the number of IPUs available. For example: on a Pod16, with a 4-IPU pipeline, the replication_factor must be between 1 and 4.
• gradient_accumulation_steps (int, optional, defaults to 1) — Number of micro-batches to accumulate for the gradient calculation. Accumulates the gradient gradient_accumulation times before updating the model using the gradient.

Parameters related to parallelism

• layers_per_ipu (List[int]) — Specifies the number of layers that will be put on each IPU for pipelined execution during training. For instance: [2, 3, 4, 2] specifies a 4-IPU pipeline, where the first two layers will be put on IPU0, the following three on IPU1, the next four on IPU2 and the last two on IPU3. If the default of [-1] is used, the layers will be split evenly over ipus_per_replica IPUs. The wildcard value ‘-1’ can also be used in combination with integers. For instance: [1, 2, -1, -1] specifies a 4-IPU pipeline, where the first layer is put on IPU0, the next two layers on IPU1, and the remaining layers split evenly between IPU2 and IPU3.
• inference_layers_per_ipu (List[int]) — Same as layers_per_ipu for inference only.
• ipus_per_replica (int, optional, defaults to len(layers_per_ipu)) — Specifies the number of IPUs to use during training. This must be consistent with the number of IPUs used in layers_per_ipu.
• inference_ipus_per_replica (int, optional, defaults to len(inference_layers_per_ipu) if ipus_per_replica==len(layers_per_ipu) else ipus_per_replica) -- Same as ipus_per_replica` but for inference only.
• parallelize_kwargs (Dict[str, Any], optional, defaults to None) — Dictionary holding kwargs used for training model calls to parallelize.
• inference_parallelize_kwargs (Dict[str, Any], optional, defaults to None) — Dictionary holding kwargs used for inference model calls to parallelize.

Parameters for memory management

• optimizer_state_offchip (bool, optional, defaults to True) — If True, uses the off-chip memory to store the optimizer state. If False, uses the on-chip memory.
• replicated_tensor_sharding (bool, optional, defaults to False) — Shards the optimizer between replicas with zero-redundancy.
• matmul_proportion (List[float] or float, optional, defaults to 0.2) — Sets the amount of temporary memory made available during training on per-IPU basis. Use this setting to control the amount of temporary memory available to operations such as:
  • convolution
  • matrix multiplication
  • embedding lookups
  • indexing operations
• inference_matmul_proportion (List[float] or float) — Same as matmul_proportion for inference only.
• enable_half_partials (bool, optional, defaults to True) — If True, sets the data type of partial results for matrix multiplication and convolution operators to float16.
• embedding_serialization_factor (int, optional, defaults to 1 if serialized_embedding_splits_per_ipu is None) — The factor to use to serialize embeddings. Nothing happens if embedding_serialization_factor = 1. For embedding_serialization_factor > 1, the torch.nn.Embedding layer is replaced with a optimum.graphcore.modeling_utils.SerializedEmbedding layer. Note: only one of embedding_serialization_factor or serialized_embedding_splits_per_ipu should be provided.
• inference_embedding_serialization_factor (int, optional, defaults to 1 if inference_serialized_embedding_splits_per_ipu is None) — Same as embedding_serialization_factor but for inference only.
• serialized_embedding_splits_per_ipu (List[int], optional, defaults to None) — Specifies the number of splits of the embedding layer that will be put on each IPU for pipelined execution. The format has to be the same as that for layers_per_ipu however wildcards are not supported. For instance: [3, 1, 0, 0] specifies how to place an embedding layer serialized into 4 sub-embedding layers across a 4-IPU pipeline. IPU-1 has 3 splits and IPU-2 has 1 split. The remaining IPUs have no sub-embedding layers. If an argument to this parameter is provided, it must:
  • be of the form List[int>=0] with atleast 1 split.
  • have the same pipeline length as ipus_per_replica
  • have splits that are consecutive with no zeros between splits e.g. [3, 0, 2, 0] is invalid
  • for generation, splits must lie entirely on the encoder or decoder portion of the pipeline. For example the 4-IPU pipeline [3, 1, 0, 0] for an encoder-decoder model can be split into [3, 1] and [0, 0], however [0, 1, 2, 0] split into [0, 1] and [2, 0] is invalid. Note: only one of embedding_serialization_factor or serialized_embedding_splits_per_ipu should be set.
• inference_serialized_embedding_splits_per_ipu (List[int], optional, defaults to None) — Same as serialized_embedding_splits_per_ipu but for inference only.
• projection_serialization_factor (int, optional, defaults to 1 if serialized_projection_splits_per_ipu is None) — The factor to use to either serialize the matmuls that are performed in the linear projection layer, or, serialize the projection layer into a set of individual linear layers that can be optionally placed on different IPUs. Nothing happens if projection_serialization_factor = 1. If projection_serialization_factor > 1, the torch.nn.Linear layer is replaced by a optimum.graphcore.modeling_utils.SplitProjection layer if serialized_projection_splits_per_ipu is provided and the linear layer’s weights are not tied to another layer. Otherwise it is replaced by a optimum.graphcore.modeling_utils.SerializedLinear layer. Note: only one of projection_serialization_factor or serialized_projection_splits_per_ipu should be set.
• inference_projection_serialization_factor (int, optional, defaults to 1 if inference_serialized_projection_splits_per_ipu is None) — Same as projection_serialization_factor but for inference only.
• serialized_projection_splits_per_ipu (List[int], optional, defaults to None) — Analogous to serialized_embedding_splits_per_ipu. Note: only one of projection_serialization_factor or serialized_projection_splits_per_ipu should be set.
• inference_serialized_projection_splits_per_ipu (List[int], optional, defaults to None) — Same as serialized_projection_splits_per_ipu but for inference only.
• recompute_checkpoint_every_layer (bool, optional, defaults to False) — If True, uses gradient checkpointing at the end of every layer. It can help to reduce the memory impact.
• explicit_ir_inference (bool, optional, defaults to False) — If True, uses experimental explicit-IR feature of PopART for inference models. This feature is only supported for inference models. For some cases explicit-IR can provide a better memory liveness schedule, reducing the peak memory during runtime.

Parameters related to host/device synchronization

• device_iterations (int, optional, defaults to 1) — Number of iterations the device should run over the data before returning to the user during training. This is equivalent to running the IPU in a loop over the specified number of iterations, with a new batch of data each time. However, increasing the number of device iterations is more efficient because the loop runs on the IPU directly.
• inference_device_iterations (int, optional, defaults to 1) — Same as device_iterations for inference.
• output_mode (str, optional, defaults to "final") — Specifies which data to return from a model. Allowed values:
  • all: returns a result for each batch.
  • sum: returns the sum of all batches.
  • final: returns the last batch.
  • default: all for inference, final for training.