question: How to get gpu id corresponding to each process in 'ddp'?
context: In ddp mode, if I use 8 gpus, then it will creates 8 processes for each gpu, if I want to create new tensor at runtime, can I use .cuda(self.trainer.root_gpu)? will it use correct gpu corresponding to its process?
answer: yes