You are viewing v0.12.0 version.
A newer version
v1.1.0 is available.
Stateful Classes
Below are variations of a singleton class in the sense that all instances share the same state, which is initialized on the first instantiation.
These classes are immutable and store information about certain configurations or states.
class accelerate.state.AcceleratorState
< source >( mixed_precision: str = None cpu: bool = False deepspeed_plugin = None fsdp_plugin = None _from_accelerator: bool = False **kwargs )
Singleton class that has information about the current training environment.
Attributes:
- device (
torch.device) — The device to use. - distributed_type (DistributedType) — The type of distributed environment currently in use.
- local_process_index (
int) — The index of the current process on the current server. - mixed_precision (
str) — Whether or not the current script will use mixed precision. If you are using mixed precision, define if you want to use FP16 or BF16 (bfloat16) as the floating point. - num_processes (
int) — The number of processes currently launched in parallel. - process_index (
int) — The index of the current process.
Singleton class that has information related to gradient synchronization for gradient accumulation
Attributes:
- end_of_dataloader (
bool) — Whether we have reached the end the current dataloader - remainder (
int) — The number of extra samples that were added from padding the dataloader