OpenPAI and PyTorch Lightning for easy multi-machine distributed training?
What is your question?
There's an AI computing platform called OpenPAI that's an interface that makes distributed multi-GPU and multi-Node setup easy. I thought this would be a perfect opportunity for PyTorch Lightning, because the docs claim it's very easy to run distributed training. However, after reading the docs, I'm still not too sure how to setup the backend. I see some things about SLURM, but I don't know much about it. I'm also not sure how that would fit into OpenPAI.
Could anyone recommend any paths to pursue, please? I realize people here may not have heard of OpenPAI before and may be in the same boat as me. But I'd like hearing from people who are more experienced with this area. Thanks!
What have you tried?
Reading the docs of both PyTorch Lightning and OpenPAI. OpenPAI supports Horovod with OpenMPI, but that doesn't seem to match up with PTL's usage of SLURM.