Hugging Face's logo
Join the Hugging Face community

and get access to the augmented documentation experience

to get started

Stateful Classes

Below are variations of a singleton class in the sense that all instances share the same state, which is initialized on the first instantiation.

These classes are immutable and store information about certain configurations or states.

class accelerate.state.AcceleratorState

< >

( mixed_precision: str = None cpu: bool = False deepspeed_plugin = None fsdp_plugin = None _from_accelerator: bool = False **kwargs )

Singleton class that has information about the current training environment.

Available attributes:

  • device (torch.device) β€” The device to use.
  • distributed_type (DistributedType) β€” The type of distributed environment currently in use.
  • local_process_index (int) β€” The index of the current process on the current server.
  • mixed_precision (str) β€” Whether or not the current script will use mixed precision, and if so the type of mixed precision being performed.
  • num_processes (int) β€” The number of processes currently launched in parallel.
  • process_index (int) β€” The index of the current process.

class accelerate.state.GradientState

< >

( )

Singleton class that has information related to gradient synchronization for gradient accumulation

Available attributes:

  • end_of_dataloader (bool) β€” Whether we have reached the end the current dataloader
  • remainder (int) β€” The number of extra samples that were added from padding the dataloader
  • sync_gradients (bool) β€” Whether the gradients should be synced across all devices