7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76164_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76165_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76166_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76167_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76168_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76169_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76170_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76171_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76172_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76173_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76174_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76175_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76176_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76177_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76178_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76179_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76187_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76188_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76189_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76190_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76191_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76192_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76193_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76194_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76211_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76212_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76213_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76214_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76215_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76216_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76217_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76218_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76390_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76392_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76393_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76394_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76515_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76517_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76518_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76519_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76895_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76896_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76897_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76898_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76899_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76900_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76907_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76908_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76909_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76910_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76911_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_76916_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_77171_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_77172_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_77174_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_77175_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_77908_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_77936_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_78003_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_78023_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_78200_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_78201_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_78205_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_78207_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_78209_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_78210_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_78216_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_78217_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_78416_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_78417_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_78418_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_78419_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_78421_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_78424_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_78430_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_78431_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_78434_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_78435_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_78439_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_78440_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_78513_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_78514_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_78515_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_78516_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_78523_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_78524_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_78525_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/__KMP_REGISTERED_LIB_78526_10016371': Operation not permitted 7: rm: cannot remove '/dev/shm/hsakmt_shared_mem': Operation not permitted 7: rm: cannot remove '/dev/shm/rocm_smi_card0': Operation not permitted 7: rm: cannot remove '/dev/shm/rocm_smi_card1': Operation not permitted 7: rm: cannot remove '/dev/shm/rocm_smi_card2': Operation not permitted 7: rm: cannot remove '/dev/shm/rocm_smi_card3': Operation not permitted 7: rm: cannot remove '/dev/shm/rocm_smi_card4': Operation not permitted 7: rm: cannot remove '/dev/shm/rocm_smi_card5': Operation not permitted 7: rm: cannot remove '/dev/shm/rocm_smi_card6': Operation not permitted 7: rm: cannot remove '/dev/shm/rocm_smi_card7': Operation not permitted 7: rm: cannot remove '/dev/shm/sem.hsakmt_semaphore': Operation not permitted 0: Successfully preprocessed all matching files. 0: Detected CUDA files, patching ldflags 0: Emitting ninja build file /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/fused_kernels/build/build.ninja... 0: Building extension module scaled_upper_triang_masked_softmax_cuda... 0: Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N) 0: Loading extension module scaled_upper_triang_masked_softmax_cuda... 0: Successfully preprocessed all matching files. 0: Detected CUDA files, patching ldflags 0: Emitting ninja build file /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/fused_kernels/build/build.ninja... 0: Building extension module scaled_masked_softmax_cuda... 0: Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N) 0: Loading extension module scaled_masked_softmax_cuda... 0: Successfully preprocessed all matching files. 0: Detected CUDA files, patching ldflags 0: Emitting ninja build file /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/fused_kernels/build/build.ninja... 0: Building extension module fused_mix_prec_layer_norm_cuda... 0: Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N) 0: Loading extension module fused_mix_prec_layer_norm_cuda... 0: Successfully preprocessed all matching files. 0: Successfully preprocessed all matching files. 0: Successfully preprocessed all matching files. 0: Successfully preprocessed all matching files. 0: Successfully preprocessed all matching files. 0: Successfully preprocessed all matching files. 1: Successfully preprocessed all matching files. 1: Successfully preprocessed all matching files. 1: Successfully preprocessed all matching files. 6: Successfully preprocessed all matching files. 2: Successfully preprocessed all matching files. 5: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 5: warnings.warn( 5: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 5: warnings.warn( 5: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 5: warnings.warn( 5: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 5: warnings.warn( 5: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 5: warnings.warn( 5: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 5: warnings.warn( 5: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 5: warnings.warn( 4: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 4: warnings.warn( 4: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 4: warnings.warn( 5: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 5: warnings.warn( 4: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 4: warnings.warn( 4: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 4: warnings.warn( 6: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 6: warnings.warn( 4: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 4: warnings.warn( 4: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 4: warnings.warn( 6: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 6: warnings.warn( 6: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 6: warnings.warn( 1: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 1: warnings.warn( 4: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 4: warnings.warn( 6: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 6: warnings.warn( 1: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 1: warnings.warn( 4: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 4: warnings.warn( 6: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 6: warnings.warn( 1: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 1: warnings.warn( 1: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 1: warnings.warn( 1: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 1: warnings.warn( 6: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 6: warnings.warn( 6: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 6: warnings.warn( 1: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 1: warnings.warn( 6: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 6: warnings.warn( 1: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 1: warnings.warn( 1: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 1: warnings.warn( 7: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 7: warnings.warn( 7: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 7: warnings.warn( 7: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 7: warnings.warn( 7: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 7: warnings.warn( 7: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 7: warnings.warn( 7: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 7: warnings.warn( 7: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 7: warnings.warn( 7: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 7: warnings.warn( 2: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 2: warnings.warn( 2: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 2: warnings.warn( 2: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 2: warnings.warn( 2: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 2: warnings.warn( 2: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 2: warnings.warn( 2: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 2: warnings.warn( 2: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 2: warnings.warn( 2: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 2: warnings.warn( 0: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 0: warnings.warn( 0: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 0: warnings.warn( 0: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 0: warnings.warn( 0: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 0: warnings.warn( 0: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 0: warnings.warn( 0: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 0: warnings.warn( 0: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 0: warnings.warn( 3: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 3: warnings.warn( 3: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 3: warnings.warn( 3: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 3: warnings.warn( 3: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 3: warnings.warn( 3: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 3: warnings.warn( 3: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 3: warnings.warn( 3: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 3: warnings.warn( 3: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 3: warnings.warn( 0: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/venv/lib/python3.9/site-packages/torch/distributed/distributed_c10d.py:429: UserWarning: torch.distributed.distributed_c10d._get_global_rank is deprecated please use torch.distributed.distributed_c10d.get_global_rank instead 0: warnings.warn( 0: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root...Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root...Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 0: 0: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 0: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root...Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 0: 0: 1: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 1: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 1: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 0: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 1: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 1: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 1: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 1: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 1: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 2: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root...Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 2: 2: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 2: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 0: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 2: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 2: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 2: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 2: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 6: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root...Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 6: 6: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 6: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 6: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 6: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 6: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 6: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 3: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root...Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 3: 3: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root...Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root...Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root...Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root...Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root...Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 3: 3: 3: 3: 3: 4: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root...Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 4: 4: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root...Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root...Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root...Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root...Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root...Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 4: 4: 4: 4: 4: 5: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root...Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root...Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root...Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root...Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root...Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root...Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root...Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 5: 5: 5: 5: 5: 5: 5: 7: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root...Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root...Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root...Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root...Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root...Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root...Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 7: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 7: 7: 7: 7: 7: 7: 0: Emitting ninja build file /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu/utils/build.ninja... 0: Building extension module utils... 0: Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N) 0: Loading extension module utils... 0: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 0: No modifications detected for re-loaded extension module utils, skipping build step... 0: Loading extension module utils... 0: Loading extension module utils... 0: Loading extension module utils... 0: Loading extension module utils...Loading extension module utils...Loading extension module utils... 0: 0: Loading extension module utils...Loading extension module utils... 0: 0: 1: Loading extension module utils... 2: Loading extension module utils... 2: Loading extension module utils... 1: Loading extension module utils... 2: Loading extension module utils... 1: Loading extension module utils... 2: Loading extension module utils... 1: Loading extension module utils... 2: Loading extension module utils... 2: Loading extension module utils... 1: Loading extension module utils... 1: Loading extension module utils... 2: Loading extension module utils... 1: Loading extension module utils... 2: Loading extension module utils... 1: Loading extension module utils... 6: Loading extension module utils... 6: Loading extension module utils... 6: Loading extension module utils... 6: Loading extension module utils... 6: Loading extension module utils... 3: Loading extension module utils... 6: Loading extension module utils... 3: Loading extension module utils... 6: Loading extension module utils... 3: Loading extension module utils... 6: Loading extension module utils... 3: Loading extension module utils... 3: Loading extension module utils... 3: Loading extension module utils... 3: Loading extension module utils... 3: Loading extension module utils... 4: Loading extension module utils... 4: Loading extension module utils... 4: Loading extension module utils... 4: Loading extension module utils... 4: Loading extension module utils... 5: Loading extension module utils... 4: Loading extension module utils... 4: Loading extension module utils... 4: Loading extension module utils... 5: Loading extension module utils... 5: Loading extension module utils... 5: Loading extension module utils... 5: Loading extension module utils... 5: Loading extension module utils... 5: Loading extension module utils... 5: Loading extension module utils... 7: Loading extension module utils... 7: Loading extension module utils... 7: Loading extension module utils... 7: Loading extension module utils... 7: Loading extension module utils... 7: Loading extension module utils... 7: Loading extension module utils... 7: Loading extension module utils... 0: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 0: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 0: No modifications detected for re-loaded extension module utils, skipping build step... 0: Loading extension module utils... 0: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 0: No modifications detected for re-loaded extension module utils, skipping build step... 0: Loading extension module utils... 0: No modifications detected for re-loaded extension module utils, skipping build step... 0: Loading extension module utils... 0: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 0: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 0: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 0: No modifications detected for re-loaded extension module utils, skipping build step... 0: Loading extension module utils... 0: No modifications detected for re-loaded extension module utils, skipping build step... 0: Loading extension module utils... 0: No modifications detected for re-loaded extension module utils, skipping build step... 0: Loading extension module utils... 2: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root...Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 2: 2: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 2: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 2: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 2: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 2: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 2: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 2: No modifications detected for re-loaded extension module utils, skipping build step... 2: Loading extension module utils... 2: No modifications detected for re-loaded extension module utils, skipping build step... 2: Loading extension module utils... 2: No modifications detected for re-loaded extension module utils, skipping build step... 2: Loading extension module utils... 2: No modifications detected for re-loaded extension module utils, skipping build step... 2: Loading extension module utils... 2: No modifications detected for re-loaded extension module utils, skipping build step... 2: Loading extension module utils... 2: No modifications detected for re-loaded extension module utils, skipping build step...No modifications detected for re-loaded extension module utils, skipping build step... 2: 2: Loading extension module utils...Loading extension module utils... 2: 2: No modifications detected for re-loaded extension module utils, skipping build step... 2: Loading extension module utils... 6: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 6: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 6: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 6: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 6: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 6: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 6: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 6: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 6: No modifications detected for re-loaded extension module utils, skipping build step... 6: Loading extension module utils... 6: No modifications detected for re-loaded extension module utils, skipping build step... 6: Loading extension module utils... 6: No modifications detected for re-loaded extension module utils, skipping build step... 6: Loading extension module utils... 6: No modifications detected for re-loaded extension module utils, skipping build step...No modifications detected for re-loaded extension module utils, skipping build step... 6: 6: Loading extension module utils...Loading extension module utils...No modifications detected for re-loaded extension module utils, skipping build step... 6: 6: 6: Loading extension module utils... 6: No modifications detected for re-loaded extension module utils, skipping build step... 6: Loading extension module utils... 6: No modifications detected for re-loaded extension module utils, skipping build step... 6: Loading extension module utils... 5: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 5: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 5: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 5: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 5: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root...Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 5: 5: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 5: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 5: No modifications detected for re-loaded extension module utils, skipping build step... 5: Loading extension module utils... 5: No modifications detected for re-loaded extension module utils, skipping build step... 5: Loading extension module utils... 5: No modifications detected for re-loaded extension module utils, skipping build step... 5: Loading extension module utils... 5: No modifications detected for re-loaded extension module utils, skipping build step... 5: Loading extension module utils... 4: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 4: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 4: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 4: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 4: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 4: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 5: No modifications detected for re-loaded extension module utils, skipping build step...No modifications detected for re-loaded extension module utils, skipping build step... 5: 5: No modifications detected for re-loaded extension module utils, skipping build step...Loading extension module utils...Loading extension module utils... 5: 5: 4: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 4: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 5: Loading extension module utils... 5: No modifications detected for re-loaded extension module utils, skipping build step... 5: Loading extension module utils... 4: No modifications detected for re-loaded extension module utils, skipping build step... 4: Loading extension module utils... 4: No modifications detected for re-loaded extension module utils, skipping build step... 4: Loading extension module utils... 7: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 7: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root...Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 7: 7: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 7: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 7: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 7: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 7: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 4: No modifications detected for re-loaded extension module utils, skipping build step... 4: Loading extension module utils... 4: No modifications detected for re-loaded extension module utils, skipping build step... 4: Loading extension module utils...No modifications detected for re-loaded extension module utils, skipping build step... 4: 4: Loading extension module utils... 4: No modifications detected for re-loaded extension module utils, skipping build step... 4: Loading extension module utils... 4: No modifications detected for re-loaded extension module utils, skipping build step... 4: Loading extension module utils... 7: No modifications detected for re-loaded extension module utils, skipping build step... 7: Loading extension module utils... 4: No modifications detected for re-loaded extension module utils, skipping build step... 4: Loading extension module utils... 7: No modifications detected for re-loaded extension module utils, skipping build step... 7: Loading extension module utils... 7: No modifications detected for re-loaded extension module utils, skipping build step... 7: Loading extension module utils... 7: No modifications detected for re-loaded extension module utils, skipping build step... 7: Loading extension module utils... 7: No modifications detected for re-loaded extension module utils, skipping build step... 7: Loading extension module utils... 7: No modifications detected for re-loaded extension module utils, skipping build step...No modifications detected for re-loaded extension module utils, skipping build step... 7: 7: Loading extension module utils...Loading extension module utils... 7: 7: No modifications detected for re-loaded extension module utils, skipping build step... 7: Loading extension module utils... 1: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root...Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 1: 1: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 1: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 1: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 1: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 1: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 1: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 1: No modifications detected for re-loaded extension module utils, skipping build step... 1: Loading extension module utils... 1: No modifications detected for re-loaded extension module utils, skipping build step... 1: Loading extension module utils... 1: No modifications detected for re-loaded extension module utils, skipping build step... 1: Loading extension module utils... 1: No modifications detected for re-loaded extension module utils, skipping build step... 1: No modifications detected for re-loaded extension module utils, skipping build step...Loading extension module utils... 1: 1: No modifications detected for re-loaded extension module utils, skipping build step...Loading extension module utils... 1: 1: No modifications detected for re-loaded extension module utils, skipping build step...Loading extension module utils... 1: 1: Loading extension module utils... 1: No modifications detected for re-loaded extension module utils, skipping build step... 1: Loading extension module utils... 3: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root...Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 3: 3: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 3: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root...Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root...Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 3: 3: 3: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 3: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 3: No modifications detected for re-loaded extension module utils, skipping build step... 3: Loading extension module utils... 3: No modifications detected for re-loaded extension module utils, skipping build step... 3: Loading extension module utils... 3: No modifications detected for re-loaded extension module utils, skipping build step... 3: Loading extension module utils... 3: No modifications detected for re-loaded extension module utils, skipping build step... 3: Loading extension module utils...No modifications detected for re-loaded extension module utils, skipping build step... 3: 3: No modifications detected for re-loaded extension module utils, skipping build step...Loading extension module utils... 3: 3: Loading extension module utils... 3: No modifications detected for re-loaded extension module utils, skipping build step... 3: Loading extension module utils... 3: No modifications detected for re-loaded extension module utils, skipping build step... 3: Loading extension module utils... 0: Using /pfs/lustrep4/users/muennighoff/.cache/torch_extensions/py39_cpu as PyTorch extensions root... 0: No modifications detected for re-loaded extension module utils, skipping build step... 0: Loading extension module utils... 0: /pfs/lustrep4/scratch/project_462000119/muennighoff/nov-2022-bettercom/Megatron-DeepSpeed/megatron/utils.py:349: UserWarning: Parameter count with the embeddings will be inaccurate with PP > 1, as the first and last stage hold several copies of the embeddings 0: warnings.warn("Parameter count with the embeddings will be inaccurate with PP > 1, as the first and last stage hold several copies of the embeddings")