Setting random seed before training
Hi,
Do I need to manually set the random seed to ensure the synchronization of state (e.g. initialized parameters) across processes when using distributed training?
No, parameters get broadcast on the first time the model performs forward. This is implemented in the PyTorch DistributedDataParallel wrapper and so applies to Lightning as well.