You are viewing v0.16.0 version.
A newer version
v1.3.0 is available.
Stateful Classes
Below are variations of a singleton class in the sense that all instances share the same state, which is initialized on the first instantiation.
These classes are immutable and store information about certain configurations or states.
class accelerate.state.AcceleratorState
< source >( mixed_precision: str = None cpu: bool = False dynamo_backend = None deepspeed_plugin = None fsdp_plugin = None megatron_lm_plugin = None _from_accelerator: bool = False **kwargs )
Singleton class that has information about the current training environment.
Available attributes:
- device (
torch.device) — The device to use. - distributed_type (DistributedType) — The type of distributed environment currently in use.
- initialized (
bool) — Whether or not theAcceleratorStatehas been initialized fromAccelerator. - local_process_index (
int) — The index of the current process on the current server. - mixed_precision (
str) — Whether or not the current script will use mixed precision, and if so the type of mixed precision being performed. - num_processes (
int) — The number of processes currently launched in parallel. - process_index (
int) — The index of the current process.
Singleton class that has information related to gradient synchronization for gradient accumulation
Available attributes:
- end_of_dataloader (
bool) — Whether we have reached the end the current dataloader - remainder (
int) — The number of extra samples that were added from padding the dataloader - sync_gradients (
bool) — Whether the gradients should be synced across all devices