Auto load NCCL when on DDP
users must remember to load NCCL when using DDP. Wondering if there's a way for us to automatically load it for the user so they don't have to remember.
Probably would happen in the trainer.init and we'd make a system call from python.
Pseudocode
# read the available modules
out = subprocess.call('module avail')

# find the nccl lib
# we also want to take into account the cuda version used to build pytorch
# so we'd probably want to find that here as well
lib = find_nccl(out)

# load
if is_not_loaded(lib):
    subprocess.call(f'module load {lib}')