How to use pytorch-lightning distributed training without SLURM?
❓How to use pytorch-lightning distributed training without SLURM?
Couldn't find anywhere a single note or tutorial on this.
For example I have just 2 node with 4 GPUs on each.
On each node environment variables required for Pytorch distributed communication are configured (see pytorch documentation).
Is this possible to train pytorch-lightning script in this setup and if so how?
you can configure your own environment variables and do your own setup.
Just override LightningModule.init_ddp_connection
https://pytorch-lightning.readthedocs.io/en/latest/lightning-module.html#lightningmodule-class
(corrected)